Technology
Data Center Silicon Photonics: Evolution and Innovation
Light in an optical fiber covers much greater distance without repeaters (and associated repeater delay) and loses far less energy in route than do electrical signals in even the best cable. Three attributes — reach, bandwidth density, and energy consumption — are critical to the future of large-scale data centers. Consequently, optical fiber, driven by plug-in interface cards, has rapidly replaced electrical connections in the top layers — the spine and leaf networks — of the largest data centers.
But major advances in silicon photonics — implementing optical components on silicon wafers using processes derived from standard IC production techniques — are poised to spread this transformation to applications which require much shorter distances. The near future will see rapid evolutionary change as silicon photonics shrink the size and power consumption of optical interfaces, allowing the next layer down in the data center network — the top-of-rack network that connects the server cards to each other and to the leaf network — to use optical fiber. This will put the top-of-rack network on track to keep up with the exponentially growing demand for bandwidth between server cards.
That is only the start. As the progress of silicon photonics continues, the evolution will lead to even more innovation, as optical interconnect becomes the medium for connections between packages on the server cards themselves.
Fabricating Optics on Silicon Wafers
These changes are all predicated upon implementing critical optical components on silicon wafers, in processes that can be mass produced in 300mm wafer fabs. This has been a core research, development and productization objective for Intel.
A central component, and the one that is critical to manufacturing in high volume, is the light source. Intel recently published work on an eight-wavelength distributed feedback (DFB) laser array that is fully integrated on a silicon wafer and delivers excellent output power uniformity and wavelength spacing uniformity. The eight-wavelength DFB array was designed and fabricated using Intel’s commercial 300 mm hybrid silicon photonics platform, which is used today to manufacture production optical transceivers in volume. The same hybrid platform is used to build on-die semiconductor optical amplifiers that boost the amplitude of light on a photonic IC (PIC).
Intel fabricates other critical components using more common 300mm wafer-processing techniques and materials. Optical silicon waveguides allow us to move light precisely around on the surface of a PIC, much as copper interconnect moves electrical signals. Microring resonators allow a particular wavelength of light to be selected out from the mixture of wavelengths in a waveguide and modulated in a transmitter or sent to a detector in a receiver.
Optical Compute Interconnect
The next step in integration, however, will prove a deeper level of innovation for data centers. It will be possible to integrate the two dies of the optical interface with compute dies on a single package. This means the compute fabric within servers can be interconnected via optical links. This is called Optical Compute Interconnect, and it will have immediate impacts.
First, it will break open the approaching pin count and power barrier that will limit the bandwidth in and out of a package. These physical limitations are beginning to constrain how much aggregate interconnect bandwidth we can sustain using electrical interfaces. But coming out of the package directly with optical fibers opens the possibility of far greater aggregate bandwidth across the package boundary. We can keep ahead of the exponentially growing bandwidth appetite of processing packages with their growing numbers of compute cores.
Second, we can benefit from the remarkable ability of optical interconnect to conserve energy. By transitioning to optical fiber, the frequency-dependent channel loss of electrical channels is eliminated. Intel Labs research roadmap targets energy efficiency improvements while increasing bandwidth per fiber — targeting 1 pJ/bit energy efficiency with a per-fiber bandwidth of 1 Tbit/s. As we look further down the research path, Intel Labs has a university research center with even more aggressive targets of 0.25 pJ/bit energy efficiency and bandwidth per fiber of 4 Tbits/s. So while we are solving an increasingly critical bandwidth problem, we are also significantly reducing the energy-per-bit cost of moving data between packages.
Innovation
But there is one more remarkable characteristic of optical interconnect that now comes into the spotlight. Once you launch a light wave into an optical fiber, it keeps going, with only slightly attenuated over distance. This means that the optical interconnect between packages on a server board needn’t stop after tens of centimeters. It can keep going for over 100 meters. This physical characteristic raises very interesting possibilities for the way data in the compute fabric may communicate across board boundaries.
From the top-of-rack network to, ultimately, server computing fabric, the silicon photonic components that Intel is researching and developing today open the opportunity for profound improvements in data center bandwidth, energy, and organization. We stand at a fascinating threshold.
For a more detailed discussion of this technology, please see two presentations at the 2022 Hot Interconnects Conference, Transitioning from Electrical to Optical I/O and Highly Integrated 4 Tbps Silicon Photonic IC for Compute Fabric Connectivity. These presentations are available on the Hot Interconnects website.
Notes & Disclaimers:
Intel technologies may require enabled hardware, software or service activation.
Your costs and results may vary.
You may not use or facilitate the use of this document in connection with any infringement or other legal analysis concerning Intel products described herein. You agree to grant Intel a non-exclusive, royalty-free license to any patent claim thereafter drafted which includes subject matter disclosed herein.
No license (express or implied, by estoppel or otherwise) to any intellectual property rights is granted by this document.
© Intel Corporation. Intel, the Intel logo, and other Intel marks are trademarks of Intel Corporation or its subsidiaries. Other names and brands may be claimed as the property of others.
