Building Fixed HW Implementations of Neural Networks (Yale, Cornell et al.)

Researchers from Yale University, Cornell University, Boston University, and NTT Research have published a technical paper on "Physical Foundation Models: Fixed hardware implementations of large-scale neural networks." The paper discusses the concept of foundation models, which are large neural networks trained on extensive datasets for various tasks like text generation, image classification, and more. It proposes building special-purpose, fixed hardware implementations of neural networks, known as Physical Foundation Models (PFMs), to enhance energy efficiency, speed, and parameter density. PFMs could potentially reduce energy consumption in AI data centers and enable AI on power-constrained edge devices. The paper outlines the benefits and challenges of implementing trillion-parameter PFMs using examples from optical computing and nanoelectronics.