Kioxia explores using flash memory as a DRAM alternative for AI workloads

Japanese memory manufacturer Kioxia is moving closer to an ambitious idea that could change how memory is used in AI systems: using NAND flash to supplement, and in some cases potentially replace, part of the role traditionally handled by DRAM.

The company is investigating a “Flash-for-DRAM” approach aimed at the rapidly growing demands of artificial intelligence workloads. The initiative is particularly focused on using flash memory as a large-capacity memory expansion layer alongside conventional system memory.

AI systems are placing unprecedented demands on memory. Modern AI models need to process huge quantities of data, while data centres are increasingly deploying large numbers of accelerators and GPUs. High-bandwidth memory (HBM) has become particularly important because of its ability to move data rapidly between memory and processors.

However, HBM and conventional DRAM are expensive and increasingly constrained by demand. The memory industry is currently experiencing significant pressure as manufacturers attempt to satisfy AI-related requirements.

Kioxia’s proposal takes a different approach. Rather than attempting to make flash memory perform exactly like DRAM, the company is looking at using flash as an additional tier of memory that can provide much greater capacity.

One possibility involves connecting flash storage through Compute Express Link (CXL), allowing it to operate as a memory-expansion resource. This could give AI servers access to considerably more memory without requiring all of that capacity to be provided by expensive DRAM.

Flash memory is considerably slower than DRAM, meaning it cannot simply replace high-speed working memory in every application. Instead, Kioxia’s approach is aimed at workloads where extremely large amounts of data need to remain accessible but do not necessarily need to reside in the fastest possible memory.

This could be particularly useful for AI applications that handle large models, datasets and other data-intensive workloads.

The technology could also help address one of the major challenges facing AI infrastructure: memory capacity. While HBM provides enormous bandwidth, its capacity remains relatively limited compared with NAND flash. Flash can provide much greater storage density at a lower cost per bit.

Kioxia’s work therefore reflects a broader movement towards developing memory hierarchies specifically designed around AI workloads rather than relying solely on traditional CPU and server architectures.

The company is not suggesting that flash will simply replace DRAM across computers and servers. Instead, the objective is to find workloads where the lower cost and much higher capacity of flash can compensate for its slower performance.

The timing is significant. AI has created a huge increase in demand for memory, with manufacturers increasingly prioritising high-value products such as HBM for data-centre applications. This has contributed to shortages and rising prices across the wider memory market.

If Kioxia can demonstrate that flash can effectively supplement DRAM in AI systems, the technology could provide data-centre operators with another way of scaling memory capacity without relying entirely on scarce high-performance DRAM.

For the wider semiconductor industry, the development highlights how the rapid growth of AI is forcing manufacturers to rethink the traditional boundaries between memory and storage.

Kioxia’s Flash-for-DRAM initiative is still an emerging technology rather than a direct replacement for DRAM. However, as AI models continue to grow, finding economical ways of providing enormous amounts of accessible memory is likely to become increasingly important.