As generative AI fashions develop extra highly effective, their power use is changing into a critical bottleneck. A brand new totally optical generative AI chip may assist by operating superior picture and video era duties at speeds and efficiencies orders of magnitude past at present’s {hardware}.
Coaching generative AI fashions requires an unlimited quantity of computing energy and power. However as demand explodes, the method of really operating the fashions to create photographs, textual content, or video—often called inference—is rapidly changing into an excellent greater drain on sources.
Video and picture era fashions are significantly power intensive. Whereas the effectivity of those fashions is continually enhancing, a 2023 examine discovered that producing 1,000 photographs utilizing a number one mannequin produced carbon emissions equal to driving a gas-powered automotive greater than 4 miles.
One promising strategy for slashing power use is photonic computing, the place processors use gentle as an alternative of electrical energy. It’s a tactic a number of well-funded startups are pursuing in earnest. However most advances have been restricted to easier duties like picture classification or textual content era.
Now, researchers from Shanghai Jiao Tong College and Tsinghua College in China have demonstrated an all-optical chip they name LightGen that’s greater than 100 occasions quicker and extra power environment friendly than a number one Nvidia GPU on duties like video and picture era.
“LightGen offers a brand new technique to bridge the brand new chip architectures to each day sophisticated AI with out impairment of efficiency and with velocity and effectivity which can be orders of magnitude larger,” the researchers write in a latest paper on the chip in Science.
A key side of the brand new design is its density. Generative fashions sometimes require tens of millions of parameters to supply high-quality outputs, however earlier photonic chips have had, at most, a number of thousand synthetic neurons. Utilizing 3D packaging, nonetheless, LightGen integrates greater than two million onto a tool measuring only a quarter of a sq. inch.
The ensuing processing increase permits the chip to work with photographs at resolutions as much as 512-by-512 pixels. Older photonic chips sometimes broke up high-resolution photographs into smaller patches to course of them. This not solely takes longer but additionally reduces a mannequin’s potential to attract statistical correlations between the completely different patches.
The researchers additionally innovated one thing known as an “optical latent house.” Generative AI fashions work, partly, by compressing high-dimensional knowledge into easier representations. This forces them to take away much less vital data and solely retain the bits which can be integral to the enter.
These condensed representations are then saved in a multi-dimensional map of ideas known as a latent house. Fashions use these representations to generate new outputs when given a immediate.
LightGen’s builders replicated this course of fully optically. Of their chip, a full-resolution picture is transmitted by way of an optical encoder made up of a number of metasurfaces—ultra-thin buildings designed to govern gentle—after which coupled into an array of optical fibers.
This course of naturally filters out higher-order knowledge, successfully condensing the data into easier representations, that are then saved within the fiber array because the optical latent house. One other set of metasurfaces on the different finish of the system, which might be switched relying on the duty, then take the output from this latent house and use it to generate high-resolution photographs.
The researchers additionally got here up with a novel coaching strategy. Right here, the chip learns probabilistic representations of coaching knowledge, which makes it doable to deal with extra advanced duties, like creating novel outputs. It is a promising growth. To this point, most photonic chips have centered on inference not coaching.
The group examined their chip on a number of demanding duties, together with the era of high-resolution photographs of animals, changing photographs into completely different inventive types, and even turning 2D photographs into 3D fashions. Notably, the chip achieved speeds and power efficiencies greater than two orders of magnitude higher than Nvidia’s A100 GPU, one of many firm’s strongest AI chips.
The brand new optical chip isn’t prepared to interrupt out of the lab simply but. It nonetheless depends on cumbersome lasers and spatial gentle modulators to generate enter indicators, and the metasurfaces central to its design are at present made with specialised processes somewhat these you would possibly discover in commonplace chip factories.
Nonetheless, with additional growth, the work suggests optical processors could possibly be a quick, energy-efficient technique to energy the cutting-edge of an more and more power-hungry AI trade.