Nature

‘RAMmageddon’ hits labs: AI-driven memory shortage is impacting science

The soaring cost and limited supply of computer memory is slowing some projects — and spurring creative approaches.
IEEE

Efficient KV Cache Spillover Management on Memory-Constrained GPU for LLM Inference

Abstract: The rapid growth of model parameters presents a significant challenge when deploying large generative models on GPU. Existing LLM runtime memory management solutions tend to maximize batch ...
Reuters

AI's memory chip champion has a value problem

LONDON, Feb 20 (Reuters Breakingviews) - Not long ago, memory chip makers were in crisis. A post-pandemic supply glut in 2023 pushed prices into freefall, wiping out operating profits across the ...
IEEE

BlockPIM: Optimizing Memory Management for PIM-enabled Long-Context LLM Inference

Abstract: Processing-In-Memory (PIM) architectures alleviate the memory bottleneck in the decode phase of large language model (LLM) inference by performing operations like GEMV and Softmax in memory.