Near-Memory AI Computing Market Size, Share & Growth Forecast (2026-2031)

The near-memory AI computing market is projected to increase from USD 55 million in 2026 to USD 710 million by 2031, reflecting a CAGR of 66.8% during the forecast period.

Near-memory computing differs from conventional AI accelerators that simply use high-bandwidth memory. The dedicated logic must execute substantive data processing next to the memory resource rather than only expand capacity, buffer transfers or manage an interface. Marvell Structera A and the SK hynix CMM-Ax collaboration illustrate this architecture through memory-adjacent processors and long-context inference demonstrations. Qualcomm has introduced a related High Bandwidth Compute roadmap for future AI inference systems.

Market Overview

Inference systems are increasingly limited by the time and energy needed to move large working sets between memory and compute devices. Attention, embeddings and long-context retrieval generate irregular or bandwidth-intensive operations that may be better performed close to memory. Near-memory engines address these tasks with processor cores, fixed-function accelerators or programmable data-processing logic associated with DRAM and memory interconnects, leaving conventional GPUs and CPUs to handle the remainder of the model.

Marvell positions Structera A as a CXL-attached near-memory accelerator with sixteen Arm processor cores and up to 200 GB/s of local memory bandwidth. Its collaboration with SK hynix combines this processing capability with the memory supplier’s modules and software stack. The resulting CMM-Ax demonstration targets long-context language-model serving, indicating how specialized offload could complement rather than replace existing data-center accelerators.

Qualcomm’s 2026 High Bandwidth Compute announcement shows another direction, bonding computing and dense memory through stacked silicon. The company expects initial commercial sampling of its first generation in 2027, meaning near-term demand is tied primarily to designs available for qualification now. XCENA and other memory-centric developers add further approaches, but customers still need to verify availability, compatibility and measurable workload gains before large deployments.

Market Trends

• Long-Context Inference Moves Processing Toward Memory

Long-context language models repeatedly inspect large KV caches and retrieve relevant information while generating answers. Transferring this data through central processor interfaces can increase latency and memory bandwidth costs, especially when many conversations or agent sessions remain active simultaneously. Near-memory architectures aim to perform selective computation at the memory subsystem and return compact results to the main accelerator.

Marvell and SK hynix demonstrated CMM-Ax for efficient long-context LLM serving in 2026. The design uses processing within the CXL memory subsystem alongside GPUs, reflecting a practical division of labor rather than a wholesale replacement of conventional inference servers. Buyers will compare sustained token latency and power consumption against the additional integration and software costs.

• CXL Creates a Deployment Path for Specialized Memory Offload

CXL interconnect technology can expose additional memory and support dedicated engines close to that memory. In a near-memory configuration, the engine performs useful filtering, compression or other AI-related processing instead of acting solely as a capacity expansion device. This distinction matters commercially because a simple CXL controller is not a near-memory accelerator.

Marvell’s Structera portfolio separates near-memory acceleration through Structera A from expansion controllers and switches. That product separation makes it possible to target high-value memory-bound workflows without assuming that the sale of an entire CXL fabric represents near-memory compute revenue. Interoperability with server firmware and memory-management software will influence enterprise adoption.

• Stacked Compute and Memory Architectures Extend the Technology Roadmap

Advanced packaging gives designers a route to shorter connections between compute logic and densely stacked memory. Unlike a conventional HBM stack whose logic base largely moves data to an external processor, purpose-built near-memory engines can execute selected heavy operations close to the storage cells.

Qualcomm introduced High Bandwidth Compute in June 2026 as a three-dimensional near-memory architecture for future inference accelerators. Its planned 2027 sampling and later product roadmap indicate substantial potential but also a commercialization gap. Enterprise customers are therefore likely to distinguish proven memory-offload cards from unreleased integrated architectures during the early forecast years.

• AI Data Pipelines Offer Secondary Demand Outside Token Generation

Embedding retrieval, recommendation ranking and complex data preparation create memory-heavy workloads even when they do not perform the main neural-network matrix multiplication. Offloading scanning, filtering, compression and sparse operations can improve overall AI service throughput by freeing CPUs and GPUs for operations where they offer greater advantages.

NeuroBlade provides near-data acceleration for analytics and machine-learning pipelines through cloud instances and PCIe hardware, while XCENA has presented MX1 computational memory for data-intensive systems. These suppliers serve adjacent workloads with varying levels of proximity to memory. Their relevance to this market depends on the share of deployments in which genuine memory-local computation supports AI workloads rather than ordinary database acceleration.

Segment Analysis

By Commercial Offering: CXL-Attached Near-Memory Compute Engines

CXL-attached near-memory compute engines account for approximately USD 28 million in 2026 and could reach USD 335 million by 2031. These products combine processing cores with attached DRAM and CXL connectivity so that memory-intensive tasks can be executed closer to data. They are a suitable initial offering for enterprise evaluation because dedicated accelerator cards can be introduced beside established CPUs and GPUs.

Deployment depends on the measurable benefit of placing targeted work in a memory subsystem. Marvell Structera A provides a concrete implementation, while SK hynix demonstrates integration into an AI-focused memory offload configuration. Software qualification, reliable memory coherency and workloads capable of avoiding substantial upstream transfers will matter as much as bandwidth claims.

By Compute Placement Architecture: CXL-Attached Processor Near DRAM

CXL-attached processors located near DRAM represent approximately 49% of 2026 revenue, while memory-module-attached logic represents another 23%. Both place processing in the memory subsystem, but their physical integration and replacement cycles differ. A CXL accelerator card can be upgraded independently, whereas tightly integrated modules may require qualified memory suppliers and specialized platform designs.

By 2031, stacked memory-adjacent chiplet architectures are expected to gain a larger share, potentially reaching 23% of market value as advanced packaging becomes more accessible. Qualcomm HBC demonstrates the planned technology direction but is not evidence of 2026 shipped volumes. Customers should consider thermal limits, serviceability and interconnect consistency when selecting a topology.

By Primary Memory Interface: CXL-Attached DDR

CXL-attached DDR systems contribute an estimated 58% of 2026 value because they offer established memory packaging and a defined connection to enterprise host servers. These products are suitable for offloading operations that require significant capacity but do not justify dedicating the central GPU to every intermediate calculation. Their addressable demand is tied to large context windows, embedding stores and analytics pipelines.

On-package stacked-memory interfaces are expected to capture a growing share of new designs through 2031. These systems also need specialized cooling, packaging and software support. Hardware memory capacity alone does not qualify as near-memory compute unless the associated logic executes AI-relevant processing.

By Hardware Delivery Format: CXL or PCIe Accelerator Cards

Cards with CXL or PCIe interfaces account for approximately 55% of the initial market. They reduce adoption friction because a customer can evaluate a dedicated processing device in existing infrastructure before committing to custom memory stacks. A supplier must nevertheless establish the specific tasks that the card accelerates and supply a practical toolchain for orchestration with host accelerators.

Integrated accelerator and chiplet assemblies could reach around 24% of value in 2031, driven by purpose-built inference architectures. Standalone IP licensing remains relevant where large semiconductor buyers incorporate near-memory engines into custom hardware.

By Near-Memory Workload: Long-Context Attention and KV Cache

Long-context processing and KV-cache operations represent roughly 30% of 2026 demand and are expected to approach 49% in 2031. These workloads repeatedly traverse large memory-resident data structures, creating opportunities to perform searching, filtering and attention-adjacent calculations near the stored information. Marvell and SK hynix have demonstrated an approach specifically aimed at long-context LLM serving.

Recommendation and embedding retrieval form another substantial segment because candidate matching and sparse lookups often require moving more data than the corresponding arithmetic would justify. Database acceleration and AI preprocessing remain relevant, but purchasing decisions will depend on proof that the offloaded work improves the complete model-serving system rather than merely an isolated memory benchmark.

By Deployment Environment: Hyperscale and Neocloud Data Centers

Hyperscale and neocloud operators are estimated to account for 45% of 2026 revenue and 58% by 2031. Such customers manage large amounts of memory-intensive inference traffic, making even modest reductions in data movement potentially valuable at fleet scale. They also have the engineering teams required to validate experimental offload engines and integrate them into existing heterogeneous clusters.

Enterprise and sovereign data centers represent around 24% of the initial market, supported by private inference and controlled data environments. Research clusters, industrial deployments and edge platforms are smaller but still relevant. Adoption will follow system certification, availability and operating economics rather than the number of memory technologies announced.

By End-User Industry: Cloud Infrastructure and AI Developers

Cloud infrastructure companies and AI model developers account for an estimated 43% of 2026 expenditure, rising to 57% by 2031. Their purchases are driven by high-frequency inference, context management and retrieval at scale. Near-memory solutions compete for budget against additional GPUs, faster conventional memory, optimized inference software and distributed caching.

Enterprise IT, digital commerce and financial-services operators provide secondary demand through search, recommendation and data-intensive application platforms. Industrial users may pursue specialized near-data processing for robotics or local analytics, although the commercial share must be distinguished from ordinary analytics acceleration. Successful suppliers will demonstrate both accelerated workloads and credible long-term support.

Market Drivers

• Growing Memory-Processing Load in LLM Inference

Long-context inference and growing KV caches increase the volume of information that central processors need to inspect. Moving appropriate scanning and filtering operations to memory-adjacent processors can reduce interface traffic and improve latency in selected workloads.

• Demand for Lower AI Infrastructure Energy Costs

Memory transfers consume energy even when the actual data operation is simple. Large cloud installations therefore value system designs that avoid repeated transfers and preserve processor capacity for complex operations. Measured power savings, rather than theoretical memory bandwidth, will determine procurement.

• Expansion of CXL-Compatible Server Platforms

CXL brings a common hardware interface to memory expansion and related offload technologies. Customers that already deploy compatible server components may be able to trial near-memory accelerators without replacing an entire AI hardware stack, lowering the integration threshold for qualified applications.

• Increasing Use of Retrieval and Recommendation Pipelines

Enterprise AI increasingly combines language models with embedding stores, vector matching, ranking and pre-processing. These functions involve data-intensive access patterns that may run efficiently near memory, allowing central inference processors to focus on neural-network computation.

• Advances in Packaging and Memory Co-Design

Three-dimensional packaging and closer semiconductor collaboration support new arrangements of logic and memory. Joint development by compute-chip and memory suppliers can improve control over bandwidth, power and software mapping, extending the addressable set of AI operations suited to near-memory execution.

Market Restraints

• Limited Production Qualification Across Suppliers

Many near-memory AI products remain in evaluation or roadmaps rather than broad production. Customers seeking dependable AI service capacity may delay purchases until vendors demonstrate stable parts, validated server configurations and service commitments.

• Difficult Workload Partitioning and Software Integration

Performance gains depend on assigning the right tasks to near-memory processors and keeping coordination overhead low. Poor partitioning can add synchronization cost, increase data replication and offset the energy benefit of reduced memory transfers.

• Competition from Conventional GPU and Memory Improvements

GPU kernel optimization, higher-bandwidth HBM, inference software improvements and larger caching systems can reduce the same bottlenecks. Near-memory products must show incremental benefits after these alternatives are considered, particularly when a customer already has substantial sunk hardware investment.

• Unclear Price-Performance Thresholds for Early Designs

The value of a near-memory engine varies sharply by model, context length, input distribution and utilization. Enterprises may find it difficult to compare specialized solutions on a common basis, slowing procurement and extending expensive proof-of-concept cycles.

• Thermal, Memory Coherency and Platform Certification Constraints

Tightly coupled logic and memory can introduce thermal management, reliability and firmware challenges. Memory vendors and server manufacturers may require lengthy compatibility testing before a solution is accepted in high-availability production environments.

Regional Outlook

North America

North America represents approximately 43% of near-memory AI computing demand in 2026 and is projected to reach USD 276.9 million by 2031. The United States concentrates the largest cloud infrastructure buyers, semiconductor design operations and specialized AI platform developers. Marvell and Qualcomm anchor important development work, while enterprise buyers evaluate potential savings in data-center power and inference latency.

Large operators are positioned to trial new memory-offload components earlier than most users because their workloads create enough repeated memory access to justify substantial testing. Adoption will depend on consistent software behavior, qualified hardware availability and a procurement case against additional GPU capacity or improvements to conventional memory.

Asia Pacific

Asia Pacific is expected to be the fastest-growing regional market, rising from about USD 15.4 million in 2026 to USD 241.4 million by 2031. South Korea benefits from SK hynix memory development and the participation of XCENA in emerging memory-centric systems. China, Japan and Taiwan contribute large AI electronics, semiconductor integration and enterprise infrastructure demand.

Regional progress depends on the transition from demonstrations to repeatable product shipments. SK hynix’s 2026 CMM-Ax demonstrations show technical potential in long-context inference, but large-scale revenue requires production qualification. Continued collaboration between memory manufacturers, custom silicon firms and system integrators could accelerate deployment, especially where customers operate dense AI inference clusters.

Competitive Landscape

Near-memory AI computing brings together established semiconductor infrastructure suppliers and smaller specialists developing memory-local acceleration. Marvell offers a defined Structera A product within a broader CXL portfolio, whereas SK hynix contributes memory devices and architectural integration around CMM-Ax. Qualcomm has presented an ambitious HBC architecture for future inference platforms, providing a potential route to tightly packaged compute and memory in large-scale systems.

XCENA is developing computational-memory products and working with selected partners, while NeuroBlade has commercial near-data acceleration products addressing analytics and AI-adjacent pipelines. UPMEM and Numem provide additional memory-processing technologies, though public evidence is more limited regarding AI-specific realized revenue. Customers will compare companies by actual shipped configurations, supported workloads, compiler and runtime integration, and power-performance evidence rather than by architectural claims alone.

Competitive differentiation is likely to shift toward complete heterogeneous-system deployment. Suppliers able to document stable workload offload, memory coherency, manufacturer support and favorable economics against existing GPU and CXL infrastructure will be better positioned to convert trials into production orders.

Recent Developments

• 17 March 2026: Marvell expanded its CXL portfolio with Structera S switching technology and identified Structera A as its separate near-memory acceleration offering.

• 1 June 2026: XCENA announced USD 135 million of Series B funding to support MX1 computational-memory development, partner qualification and commercialization.

• 24 June 2026: Qualcomm unveiled High Bandwidth Compute, a near-memory inference architecture with first-generation sampling expected in 2027.

• 5 August 2026: Marvell and SK hynix described their collaboration around Structera A and CMM-Ax for memory-local processing in long-context AI inference.

• August 2026: SK hynix demonstrated CMM-Ax long-context LLM serving at Future of Memory and Storage, highlighting potential performance benefits of memory-side computation.

• 12 October 2026: Marvell introduced its next-generation Structera X memory-expansion controllers, strengthening the adjacent CXL ecosystem but not constituting a new near-memory compute shipment.

Market Outlook

Near-memory AI computing is expected to develop from a small number of qualified memory-offload configurations toward more integrated forms of data-local inference acceleration. Initial adoption will center on CXL-attached processors and selected memory-intensive AI data pipelines, where existing systems provide a practical test environment. Longer-term opportunities include compute-enabled memory modules and highly integrated chiplet designs for increasingly demanding context-processing workloads.

The transition from technical demonstration to recurring purchases will remain uneven. Established suppliers can draw on broader server and memory ecosystems, while new entrants must demonstrate practical software interoperability and reliable supply. Hardware selection will be driven by full-system token latency, throughput, energy use and supportability, not by isolated chip bandwidth figures.

North America is expected to retain the largest revenue share through 2031, while Asia Pacific grows more quickly as memory design and semiconductor manufacturing ecosystems advance. Commercial success will depend on how rapidly vendors can establish workloads where near-memory processing offers a compelling advantage over optimized conventional systems.

Companies Profiled

• Marvell Technology, Inc.

• SK hynix Inc.

• QUALCOMM Incorporated

• XCENA Inc.

• NeuroBlade Ltd.

• UPMEM

• Numem Inc.

• INTERA Group

Near-Memory AI Computing Market Scope: