Credit: Thomas Fuller/NurPhoto via Getty Images
DeepSeek — a slimmer, lighter LLM that required way less data center energy to do its job and performed surprisingly well on benchmark tests against heftier American AI models. (Ironically, it was built atop an open source U.S. model, Meta’s Llama).
DeepSeek may have foundered on privacy concerns, but the trend towards smaller and smarter AI isn’t going away. The evolution is on display again in TurboQuant, a compression algorithm that Google quietly unveiled this week via a Google Research paper.
The paper itself is pretty impenetrable if you’re not an AI nerd who talks tokens and high-dimensional vectors. We’ll get into a more detailed explanation below. But here’s the TL;DR: The TurboQuant algorithm can make LLMs’ memory usage six times smaller.
What does that mean? Less energy usage, perhaps to the point where running a powerful AI model on your powerful smartphone becomes possible. Less RAM usage, right on time for the ongoing RAM shortage.
Certainly, algorithms like this can help LLMs make more efficient use of the data centers they’re hosted in — either by using the extra space to run more complex models, or, hear me out, by allowing us not to rush into building so many unpopular new data centers in the first place.
And that, paradoxically, could be a problem for the AI economy, at least as it’s currently structured.
Why smaller and smarter will mess up NVIDIA
For the past three years, tech stocks have been riding ever higher on the back of one company alone: NVIDIA. And NVIDIA has been riding ever higher on the assumption that we’re in the middle of what CEO Jensen Huang called this month “the largest infrastructure buildout in history” — an explosion of data centers, for which NVIDIA will be the chief provider of chips.
But that infrastructure build-out, if you look at data centers actually built versus data centers promised, is already stumbling, as a fresh New York Times investigation just made clear. What’s the holdup? Not just opposition from concerned citizens across the U.S., now including the NAACP. It’s also permits, applications, inspections, and the other unsexy but often necessary parts of the local government machinery.
Not least of the problems: A dearth of power generation