Korea Economic Daily: As AI CPUs Eat Up DRAM – The Memory “Shortage” Will Last Another Year!

Driven by a surge of over 100% in commodity DRAM prices, the memory industry has achieved record-breaking results. With CPUs specifically designed for AI now in use, forecasts indicate the shortage is expected to persist for another year.

Intel's recently launched "AI CPU" is expected to handle up to four times more commodity DRAMthan previous generations. Together with the surge in demand for high-capacity DRAM by GPUs, observers expect memory supply capabilities of Samsung Electronics and SK Hynix will not be sufficient to meet demand.According to industry sources on the 2nd,CPU manufacturers are seeking to integrate 300–400GB of DRAM into AI CPUs. This is an eye-popping scale, four times larger than typical CPU products (96–256GB).CPUs Emerge as "AI Coordinators"The explosive demand for high-capacity DRAM for AI CPUs is closely tied to the industry's shift towards inference-centric structures. In the past, AI inference was limited to simple Q&A, but now CPUs have become "coordinators" that manage various agent-type AIs.The key in this process is "contextual memory." In order for CPUs to coordinate entire workflows by referencing the outputs of each agent-type AI, they must be able to remember information. As a result, expanding memory—i.e., storage capacity—becomes crucial.So far, AI data centers have built computing infrastructures centered around GPUs equipped with high-bandwidth memory (HBM). Leveraging the GPU’s advantage in training AI with massive datasets, the focus has primarily been on “AI training.” Therefore, server configurations have followed an 8-GPU to 1-CPU pattern. However, as the industry shifts towards inference, server setups with much higher CPU ratios are proliferating.In a recent earnings call, Intel executives explained: “In AI inference infrastructure, the compute structure has shifted to a 1:4 ratio of CPUs to GPUs, and this trend is moving further towards 1:1.”After GPUs, CPUs Join the Battle for Memory – Demand SnowballsThe battle for memory capacity has now expanded from GPUs to CPUs, and the scale is snowballing. NVIDIA’s next-generation AI chip “Vera Rubin” comes with 288GB via 8 HBM stacks, while AMD’s next-gen GPU MI400 features an even larger 432GB capacity.Google’s recently launched custom chip—the eighth-generationTensor Processing Unit(TPU 8i)—is also scheduled to have 288GB of HBM. Additionally, Intel’s AI CPU “Xeon” and AMD’s “EPYC” are beginning to use high-capacity DDR5 of up to 400GB, and the memory shortfall is expected to last longer.Market fervor is already shown in spot prices. According to Kiwoom Securities, while the price of the older DDR4 (16GB benchmark) plunged 16% in April, the spot price of DDR5 (16GB benchmark) for AI CPUs rose 2.8% over the same period, maintaining its price premium.An industry insider stated:“The current DRAM market is estimated to be about 10 percentage points below demand. On top of HBM, the boom in commodity DRAM demand means the supercycle is very likely to be extended from the previously anticipated 2026 to 2027.”