Why Vision LLMs Force A Rethink Of Edge AI Hardware

Vision-centric large language models (LLMs) are changing the landscape of edge AI hardware, requiring a shift in architecture to accommodate real workloads, memory behavior, and sustained utilization. Traditional edge AI silicon optimized for convolutional networks is no longer sufficient as multimodal models become prevalent. Running Vision LLMs on-device offers benefits like reduced latency and improved privacy but poses challenges related to memory traffic and utilization. To address these challenges, a more realistic optimization stack is needed, focusing on model architecture, system-level scheduling, and dedicated hardware support. Dedicated hardware support is crucial for sustaining utilization across real multimodal graphs and controlling external memory traffic effectively.