Researchers at Peking University have developed an all-optical interconnect system that increases AI distributed inference speed by more than 100 times [1].
This development addresses the critical energy and hardware bottlenecks facing modern artificial intelligence. As AI models grow in complexity, the demand for computational power and electricity has surged, making traditional electronic data transfers a primary point of inefficiency.
The new system links standard electronic chips using light rather than electricity to move data. By utilizing an all-optical interconnect, the researchers achieved a speed increase of more than 100-fold [1]. This acceleration occurs during the inference phase, which is the process where a trained AI model applies its knowledge to provide a result or prediction.
Beyond speed, the system significantly lowers the overhead required for these operations. The researchers said that the compute resources required for this process were reduced to one-ninth of typical usage [1]. This reduction in power suggests that high-performance AI could eventually operate with a much smaller hardware footprint.
The project was conducted in Beijing, China, at Peking University [1]. The team focused on creating a bridge between existing electronic chip architecture and optical technology to ensure the system could be integrated into current distributed AI environments.
By moving data with light, the system avoids the heat and resistance associated with copper wiring. This shift allows for faster communication between the multiple chips that typically work together to run large-scale AI models [1].
“AI distributed inference speed increased by over 100 times”
This breakthrough represents a shift toward hybrid optoelectronic computing. By decoupling the processing (electronic) from the communication (optical), the researchers have mitigated the 'memory wall' and energy inefficiencies that currently limit the scaling of large language models. If scalable, this technology could reduce the operational costs of AI data centers and decrease the reliance on massive power grids to sustain AI growth.
