RPU: A Chiplet-Based Architecture To Address The Challenges of the Modern Memory Wall (Harvard University)

Researchers from Harvard University have introduced the Reasoning Processing Unit (RPU) to address the challenges posed by the modern memory wall. The RPU is a chiplet-based architecture that includes a Capacity-Optimized High-Bandwidth Memory (HBM-CO), a scalable chiplet design prioritizing bandwidth, and a decoupled microarchitecture to enhance memory utilization. Simulation results demonstrate significant improvements in latency and throughput compared to existing systems. The RPU aims to optimize memory bandwidth for emerging reasoning large language model (LLM) applications, enhancing system performance and energy efficiency.