Impact Of On-Chip SRAM Size And Frequency On Energy Efficiency And Performance of LLM Inference (Uppsala Univ.)

Researchers at Uppsala University published a technical paper titled “Prefill vs. Decode Bottlenecks: SRAM-Frequency Tradeoffs and the Memory-Bandwidth Ceiling,” exploring the impact of on-chip SRAM size and operating frequency on the energy efficiency and performance of Large Language Models (LLM) inference. The study focuses on the behaviors of the compute-bound prefill and memory-bound decode phases, finding that total energy use is mainly determined by SRAM size in both phases. The research suggests that an optimal hardware configuration for energy-efficient LLM accelerators includes high operating frequencies (1200MHz-1400MHz) and a small local buffer size of 32KB to 64KB. Additionally, the study highlights the role of memory bandwidth as a performance ceiling and provides insights for designing energy-efficient LLM accelerators, particularly for data centers aiming to reduce energy overhead.