Back to home
Technology

Silicon Photonic Interconnected Chiplets With Computational Network And IMC For LLM Inference Acceleration (NUS)

Source

SemiEngineering

Published

TL;DR

AI Generated

Researchers at the National University of Singapore published a technical paper titled “PICNIC,” detailing a 3D-stacked chiplets-based large language model (LLM) inference accelerator. The system includes non-volatile in-memory-computing processing elements and an Inter-PE Computational Network interconnected via silicon photonics to address communication bottlenecks. The paper introduces a LLM mapping scheme for optimizing hardware scheduling and workload mapping, achieving significant speedup and efficiency improvements over Nvidia A100. The system further scales efficiently with chiplet clustering and power gating scheme implementation, outperforming Nvidia H100 in efficiency.

Read Full Article

Similar Articles

Quantum photonics roadmap — how Xanadu and PsiQuantum are looking to transfer qubits through beams of light

Quantum photonics roadmap — how Xanadu and PsiQuantum are looking to transfer qubits through beams of light

Quantum photonics is explored as a method for transferring qubits through light beams, focusing on Xanadu and PsiQuantum. Xanadu, aiming for a Nasdaq listing, uses continuous-variable encoding, while PsiQuantum employs a dual-rail encoding approach. Quantum photonics offers advantages like room temperature operation and less susceptibility to interference but faces challenges such as photon loss. Xanadu and PsiQuantum are working towards fault-tolerant quantum computing, with Xanadu demonstrating advancements in GKP qubits and PsiQuantum aiming for a 1 million-plus qubit design. Both companies are navigating the complexities of quantum mechanics and the capital-intensive nature of quantum technology development.

Tom's Hardware
SemiEngineering

Panel-Level Packaging’s Second Wave Meets Engineering Reality

Panel-level packaging is gaining traction due to economic pressures and the increasing size of AI accelerators and HPC packages. Glass substrates are being explored to address warpage and dimensional stability issues, but they introduce new failure modes that require material solutions. Challenges in panel-level processing include materials and process integration, not just packaging problems. The industry is moving towards panels driven by economic and technological shifts, but solving these challenges requires a holistic approach.

SemiEngineering
SemiEngineering

Chiplet Standards Aim For Plug-n-Play

Chiplet standards are crucial for creating a marketplace where chiplets can be easily interchanged like LEGOs. Various standards are being developed to ensure interoperability and physical composability of chiplets, including die-to-die interconnect standards like Bunch of Wires (BoW) and Universal Chiplet Interconnect Express (UCIe). These standards cover system architecture, security, power delivery, data semantics, physical placement, testing, and more. Organizations like the Open Compute Project (OCP) are leading efforts to standardize chiplet-related aspects, such as packaging descriptions and system architectures. The goal is to pave the way for a plug-and-play chiplet marketplace, although challenges related to practical and economic factors still exist.

SemiEngineering
SemiEngineering

Silicon Photonics Lights The Way To More Efficient Data Centers

Silicon photonics is paving the way for more efficient data centers by potentially increasing bandwidth density and reducing power consumption, particularly driven by AI workloads. However, challenges such as process compatibility, thermal issues, and mechanical stress arise due to the use of various materials in photonic interconnects. Integrated electro-optical I/O modules are the desired outcome, but design and process complexities need to be addressed. The article delves into the technical aspects of silicon photonics, including the components involved, such as light sources, modulators, waveguides, and photodetectors, as well as the challenges of integrating optical and electronic components for efficient data transmission.

SemiEngineering

We use cookies

We use cookies to ensure you get the best experience on our website. For more information on how we use cookies, please see our cookie policy.