1. SK hynix proposes pooled memory for AI servers
SK hynix presented a shared memory pool concept for AI servers at the Intel AI Summit in Seoul on November 12, 2026, proposing that high-capacity memory move out of individual machines and into dedicated racks reachable by many GPU and CPU servers. In SK hynix CXL memory tests, the approach cut time to first token from 80.76 seconds to 15.45 seconds.
The South Korean memory maker is developing the idea with Intel and other industry partners. The goal is to replace the traditional model, in which every server carries its own private memory, with a common pool that multiple machines can draw on as their workloads demand.
The proposal arrives as the AI industry grapples with a growing memory squeeze. Agentic AI, software that carries out multi-step tasks on its own rather than answering a single prompt, places heavy and rising demands on both memory capacity and memory bandwidth. Processing power has advanced rapidly, but feeding that power with enough fast memory has become one of the main limits on how efficiently AI systems run.
2. What is CXL memory pooling?
CXL memory pooling is a way of letting several servers share one large reservoir of memory over a high-speed link, so that memory no longer has to be fixed inside each machine. SK hynix's design relies on Compute Express Link (CXL) and Remote Direct Memory Access (RDMA) to make this possible.
CXL is an interconnect standard that allows processors to use external memory almost as if it were installed locally. A CPU or accelerator can read from and write to a CXL-attached memory device without the software needing to treat it as slow, distant storage. RDMA complements this by letting one machine read another machine's memory directly, without routing every request through the main processor of the target system. That reduces overhead and keeps latency low when data has to cross from one server to another.
Combined, the two technologies allow memory to be placed in its own racks and shared across a cluster. SK hynix said these pooled racks would sit alongside existing server setups rather than replace them, meaning data center operators could add shared capacity without tearing out the infrastructure they already run.
3. Why per-GPU memory is becoming a bottleneck
In today's AI servers, high-bandwidth memory (HBM) is attached directly to each GPU. HBM is extremely fast, which is why it has become essential for training and running large models, but the amount fitted to each accelerator is fixed when the hardware is built.
That rigid arrangement creates two problems. When a workload needs more memory than a single GPU carries, the system must either spread the job across more accelerators or discard and later rebuild data, both of which waste time and resources. And when a GPU is not using all of its memory, that spare capacity sits idle, unavailable to neighboring machines that might need it.
SK hynix has argued that this design is reaching its limits as AI workloads change. The company credited joint work with partners across the ecosystem for progress on an alternative that treats memory as a shared resource rather than a component locked to each chip.
4. The KV cache problem the design targets
The shared memory pool is aimed squarely at the KV cache, the working memory an AI model builds up as it processes a conversation or task. KV stands for key-value, a reference to the internal data structures a model stores so it does not have to recompute earlier parts of a conversation each time it generates a new word.
The KV cache grows quickly during multi-turn inference, the back-and-forth exchanges typical of chat assistants and AI agents. Every additional turn adds context that the model must keep on hand. For agentic systems that run long chains of steps, the cache can swell to a size that no longer fits comfortably in the memory attached to a single GPU.
When that happens, systems typically fall back on a technique called eviction and recompute. Older parts of the cache are dropped to free up space, and the model rebuilds them from scratch when they are needed again. The approach keeps the system running but adds significant delay, because the GPU spends time redoing work it had already completed.
5. SK hynix CXL memory test results
In SK hynix's internal testing, the difference between the two approaches was large. Using eviction and recompute, the average time to first token, the delay before a model begins producing its response, was 80.76 seconds.
When the KV cache was instead held in remote CXL DRAM drawn from a shared pool, the same measure fell to 15.45 seconds. That represents a reduction of roughly 81%, or a little more than five times faster.
Time to first token is a key metric for interactive AI services. Users notice long pauses before a reply begins, and for agents that chain many model calls together, delays at each step compound into much longer task times. Cutting that wait directly improves both the experience for users and the number of requests a given set of hardware can serve.
The comparison also illustrates the underlying trade-off. Fetching a stored cache from shared memory across a network link is slower than reading it from memory attached to the GPU, but it can be far faster than discarding the cache and rebuilding it from scratch. The test suggests that, for long multi-turn sessions, the cost of reaching across the rack is small compared with the cost of recomputation.
The figures come with a clear caveat. They are results from SK hynix's own testing, not independent benchmarks, and real-world performance will depend on the specific models, workloads and system configurations involved.
6. Validation with Intel and customers
SK hynix said the architecture will be validated together with Intel, customers and other strategic partners, with that work starting at the system-design phase. Bringing partners in at the design stage, rather than after hardware is finalized, is intended to make sure the pooled memory works smoothly with the processors, interconnects and software that will surround it.
That step matters because memory pooling touches nearly every layer of a data center. Operating systems, AI frameworks and orchestration software all need to understand how to place data in shared memory and retrieve it efficiently. Without that support, the theoretical benefits of a shared pool would be difficult to realize in production.
The concept therefore remains at an early stage. The Seoul presentation laid out the architecture and the internal results, but the path to commercial deployment runs through a validation process whose timeline the company did not specify.
7. Building on an existing Intel partnership
The memory pool project extends a working relationship between SK hynix and Intel. In December 2025, SK hynix received Intel Data Center certification for a 256GB DDR5 RDIMM, a high-capacity server memory module, on Intel's Xeon 6 platform.
That module delivered up to a 16% gain in inference performance and cut power consumption by about 18%, according to the company. Gains of that kind are significant for data center operators, for whom electricity and cooling are among the largest ongoing costs of running AI infrastructure.
The certification established that SK hynix memory could meet Intel's requirements for server deployments. The shared memory pool takes that cooperation a step further, from validating individual modules to jointly designing how memory is organized across entire racks.
8. A memory-centric vision for agentic AI
The proposal fits a broader strategy at SK hynix centered on memory-centric computing, an approach that treats memory as a first-class part of system design rather than an add-on to processors.
For most of computing history, system design has started with the processor, with memory sized and placed to serve it. Memory-centric thinking reverses that emphasis for workloads where moving and storing data, rather than raw arithmetic, determines overall speed. AI inference with large contexts is increasingly one of those workloads, which is why memory suppliers see an opening to shape how future systems are assembled.
In the architecture the company described, each component takes on a distinct role. Intel CPUs orchestrate agentic AI workloads, coordinating the many steps and tools an agent uses. GPUs handle the heavy computation of running the models themselves. The pooled memory holds the steadily growing context, including the KV cache, so that it remains available to whichever machine needs it next.
The division of labor reflects how AI workloads are shifting. Training giant models remains compute-intensive, but serving those models to large numbers of users, especially through agents that run long multi-step sessions, is increasingly limited by how much context can be stored and how quickly it can be accessed. A design that lets memory scale independently of GPUs addresses that constraint directly.
For SK hynix, one of the leading suppliers of HBM, the push also broadens its role in AI infrastructure. Rather than competing only on the memory attached to each accelerator, the company is positioning itself to supply the shared memory layer that could sit across entire clusters.
9. What comes next for shared memory pools
The next milestone is the validation work with Intel, customers and strategic partners. Results from that process will show whether the gains seen in internal testing hold up across real deployments and a wider range of models.
Data center operators will also be watching for signs of software support and for details on how pooled memory racks would be priced and integrated alongside existing servers. Those practical questions will determine how quickly the concept can move from a conference presentation to production use.
For now, SK hynix has made the case that sharing memory across machines could ease one of the most pressing bottlenecks in AI inference. Whether that idea reshapes how AI servers are built will depend on how the design performs once it leaves the lab.