Microsoft Research has unveiled an analog optical computer (AOC) with the potential to revolutionize the efficiency of running large language models and tackling complex mathematical problems. This development stems from a four-year project within Microsoft’s Future AI Infrastructure initiative, aimed at creating novel hardware solutions for AI and cloud data centers.
On September 3rd, the team announced two significant achievements: the development of an optimization-solver algorithm and the creation of an analog digital twin of the AOC.
Hitesh Ballani, a researcher in future AI infrastructure at Microsoft Research Cambridge, UK, emphasized the real-world impact of this technology. “We have actually delivered on the hard promise that it can make a big difference in two real-world problems in two domains, banking and healthcare,” Ballani stated in a press release. “We opened up a whole new application domain by showing that exactly the same hardware could serve AI models, too.”
The AOC project, initiated four years ago, significantly enhances the capabilities of optical data transmission. While optical cables have been used for data transfer for half a century, Microsoft’s innovation increases the number of parameters or weights handled from 64 to 256.
The developed optimization-solver algorithm has demonstrated its utility in addressing persistent challenges in the banking sector. Barclays Bank PLC utilized the algorithm to tackle intricate delivery-versus-payment securities problems. The AOC successfully resolved a transaction settlement scenario involving up to 1,800 hypothetical parties and 28,000 transactions. While this is only a fraction of the volume typically processed by clearinghouses, it demonstrates the potential of the technology.
Moreover, the algorithm’s capabilities extend to the medical field, where it can accelerate MRI scans. A process that typically takes 30 minutes can potentially be reduced to five minutes using the AOC.
“To be transparent, it’s not something we can go and use clinically right now,” clarified Michael Hansen, senior director of biomedical signal processing at Microsoft Health Futures. “Because it’s just this little small problem that we ran, but it gives you that little spark that says, ‘Oh boy! If this instrument was actually in full scale’ …”
Microsoft Health Futures employed a digital twin of the AOC to validate the machine’s feasibility and to scale its capabilities beyond the limitations of the current instrument.
Francesca Parmigiani, a Microsoft principal research manager and the head of the AOC development team, characterized the computer as specialized, but highly effective. The AOC is “not a general purpose computer, but what we believe is that we can find a wide range of applications and real-world problems where the computer can be extremely successful,” said Parmigiani.
A significant advantage of the AOC lies in its potential to reduce the energy consumption associated with large language model (LLM) reasoning, specifically state tracking. The AOC achieves this by employing a fixed-point search computation method, which iteratively seeks a solution to a problem.
The AOC operates by performing computations based on varying light intensities that pass through a digital sensor. The researchers constructed projectors using optical lenses, digital sensors, and micro-LEDs. They prioritized the use of commercially available components, such as smartphone camera sensors, to minimize costs and ensure manufacturability within existing supply chains.
Microsoft Research plans to develop a new generation of the AOC every two years.
“To have the kind of success we are dreaming about, we need other researchers to be experimenting and thinking about how this hardware can be used,” Parmigiani emphasized.
The team’s findings have been published in the journal *Nature*, and a repository for the digital twin of the AOC is available on GitHub, encouraging further research and development in this field.
