The AI market is urgently demanding a next-generation solution to the interconnection bottleneck that traditional copper interconnections cannot overcome. Broadcom appears to be working on an optics-based solution that is closer to the chip itself. The company is developing new silicon photonics technology aimed at significantly increasing the bandwidth available to GPUs and other AI accelerators.
By utilizing co-packaged optics (CPOs), Broadcom aims to integrate optical connectivity components directly into GPUs, enabling higher data rates while simultaneously reducing power requirements. The company has been working on CPO solutions for several years and showcased its latest advancements at the recent Hot Chips convention.
Broadcom’s “optical engine” reportedly delivers a total interconnect bandwidth of 1.6 TB/sec. This new connection can provide error-free data transfer to a single chiplet, achieving performance levels comparable to Nvidia’s NVLink and other specialized data center solutions. The company utilized TSMC’s chip-on-wafer-on-substrate (CoWoS) packaging technology to bond a pair of high-bandwidth memory stacks to the compute die.
Manish Mehta, Broadcom’s vice president of the optical systems division, explained that copper connections start to degrade after just five meters, making optical communications a necessary solution for high-bandwidth applications.
Broadcom’s recent innovation in integrating optical connectivity into GPUs has the potential to revolutionize the field of AI acceleration. The company’s new silicon photonics technology, which utilizes co-packaged optics (CPOs), enables higher data rates while reducing power requirements. This development addresses the growing need for faster data transfer rates in AI workloads, where traditional copper interconnections are becoming increasingly inadequate.
As data transfer rates continue to increase, copper interconnections are struggling to keep up. Broadcom’s vice president of optical systems, Manish Mehta, notes that copper connections begin to degrade after just a few meters, highlighting the need for a more efficient solution.
Broadcom’s optical engine delivers a total interconnect bandwidth of 1.6 TB/sec, demonstrating the potential for optical interconnections to support next-generation AI workloads. While the technology is still in its early stages, the company’s advancements in CPO technology could provide a competitive advantage for AI acceleration.