British inference chip developer Fractile raises $720 million in funding

Fractile Ltd., a British startup developing artificial intelligence inference chips, has raised $720 million in funding.

Reuters reported Today it was announced that the Series C round was led by Thrive Capital, Founders Fund, Lightspeed and Redpoint. They were joined by about a dozen others, including the UK Sovereign AI Fund, the British Business Bank and Alphabet Inc’s investment arm CapitalG. Fractile is now worth $7 billion.

Fractile was founded in 2022 by AI researcher Walter Goodwin (pictured), who serves as managing director. The company allegedly expects its chips to perform up to 100 times better than current silicon chips. Additionally, Fractile promises a 90% reduction in energy consumption. The chips deliver these improvements by eliminating the need for AI models to move data to off-chip HBM memory.

Large language models generate quick answers through a workflow that involves numerous steps. When an LLM completes a processing step, it sends the results to an external HBM memory chip. These results are then retrieved to lay the foundation for the next step. These repeated rounds of data take a lot of time, which slows down the conclusion.

Fractile is not the only AI chip vendor addressing this bottleneck. Cerebras Systems Inc., which went public in May, offers wafer-sized accelerators that reduce data movement by storing information in a large onboard cache. This cache is faster than off-chip HBM modules. The company’s latest processor, the WSE-3T, can store 44 gigabytes of data.

Cerebras’ on-chip cache is based on SRAM, a type of memory that has historically only been built into processors in small quantities. SRAM cells, the basic building blocks of the technology, each consist of six transistors. These transistors read and write data faster than the single transistor in HBM cells. At the same time, the speed of SRAM comes with certain compromises: it is expensive and has limited capacity.

Fractile takes a different approach to reducing data movement. Instead of integrating SRAM into its inference chips, the company does this Stack Above this is standard DRAM memory. Placing a memory module directly above a processor’s logic circuitry minimizes the distance that data must travel. This speeds up traffic while reducing the amount of electricity needed to get there and back.

Samsung Electronics Co. is developing similar technology. In August, the company introduced an HBM variant called zHBM that can do this stacked on AI accelerators. It uses tiny copper wires to transfer data to and from the host chip. Samsung says the technology will be eight times faster than HBM5, a future version of HBM that is more powerful than versions used today.

Fractile, in turn, estimates that LLM can process several thousand tokens per second with its chips. The plan is to ship the processors as part of rack-scale systems that also include custom software. Many AI chip manufacturers offer custom compilers, software tools that automatically convert customers’ LLMs into a format compatible with their chip.

Fractile expects to begin shipping its chips next year.

Photo: Speed ​​of light