Nvidia shrinks LLM memory 20x without changing model weights

Nvidia's KV Cache Transform Coding (KVTC) compresses LLM key-value cache by 20x without model changes, cutting GPU memory costs and time-to-first-token by up to 8x for multi-turn AI applications.
Semiconductor Engineering

Developing ReRAM As Next Generation On-Chip Memory For Machine Learning, Image Processing And Other Advanced CPU Applications

In modern CPU device operation, 80% to 90% of energy consumption and timing delays are caused by the movement of data between the CPU and off-chip memory. To alleviate this performance concern, ...
EDN

Analog memory embedded in neural net processing SoCs

A new breed of system-on-chips (SoCs) serving speech recognition, voice-print recognition, and deep speech noise reduction is starting to employ analog in-memory computing solutions for simultaneously ...
Semiconductor Engineering

Will In-Memory Processing Work?

The cost associated with moving data in and out of memory is becoming prohibitive, both in terms of performance and power, and it is being made worse by the data locality in algorithms, which limits ...
Santa Clara University

Signal Processing and Machine Learning

Signal processing algorithms, architectures, and systems are at the heart of modern technologies that generate, transform, and interpret information across applications as diverse as communications, ...