AI chip crisis: Startups compete for Nvidia’s vital component

The artificial intelligence revolution is underway, but growing demand for its most crucial component has startups wondering how they can deliver on AI’s promise.

The lifeblood of Generative AI is a book-sized semiconductor known as a graphics processing unit (GPU), built by a company, Nvidia.

Nvidia CEO and founder Jensen Huang made a wild bet years ago that the world would soon be clamoring for a powerful chip usually used to create video games, but one that could also develop AI.

No company working with the generative AI models fueling the current frenzy can take off without Nvidia’s unique product: the latest model is the H100 and accompanying software.

This painful reality is one that Amazon, Intel, AMD and others are working to remedy with their own alternatives, but those attempts could take years.

“Not many GPUs”

And while the biggest tech companies pour all their financial muscle into generative AI, the smaller fish must go hunting for Nvidia’s holy grail.

“All over the world, it’s becoming very difficult to get thousands of GPUs because all these big companies are investing billions of dollars and stockpiling GPUs,” said Fangbo Tao, co-founder of Mindverse.AI, a startup based in Singapore.

“There aren’t a lot of GPUs,” he said.

Tao spoke to AFP at the TechCrunch Disrupt conference in San Francisco, where AI startups were scrambling to make their pitches to Silicon Valley venture capitalists (VCs).

ChatGPT took the world by storm just as Silicon Valley was caught in a nasty pandemic hangover as investors threw money at startups, convinced that life had gone smoothly. irreversibly online.

This proved far-fetched, and the US tech scene entered a recession with rounds of layoffs and a drying up of venture capital money.

Thanks to AI, some of the old mojo is back, and anyone with those two letters on their resume will likely see a red carpet rolled out on legendary Sand Hill Road, home to Silicon Valley’s most high-profile investors.

But as startups walk away with their venture capital money, the money in their pockets will quickly flow back to Nvidia for GPUs, either directly or through suppliers, to make their GPU dreams a reality. AI.

“We use a lot of big cloud providers (Microsoft, AWS and Google) and they all tell us they’re even having trouble getting supplies,” said Laurent Daudet, CEO of AI startup LightOn.

The problem is most acute for companies involved in training generative AI models, which require power-hungry GPUs to operate at full capacity to process quantities of data ingested over the Internet.

The computing needs are so enormous that only a few companies can raise the funds to create one of these large, cutting-edge language models.

“Suck out the oxygen”

Microsoft’s ten billion dollar investment in OpenAI is widely believed to be paid in the form of credits for access to purpose-built data centers powered by Nvidia GPUs.

Google has built its own models, and Amazon announced Monday that it was injecting $4 billion into Anthropic AI, another company that trains AI.

Training on this mountain of data currently “sucks almost all the oxygen out of the GPU market,” said Said Ouissal, CEO of Zededa, a company working to make AI less power-intensive.

“You’re looking at the middle of next year, maybe the end of next year, before you can actually receive delivery of new orders. The shortage doesn’t appear to be easing,” added Wes Cummins, CEO of ‘Applied Digital, a company that provides AI infrastructure.

Companies on the front lines of AI also point out that Nvidia’s pivotal role makes it the de facto maker of the technology’s direction.

The market is “almost entirely driven by the big players – Google, Amazon, Metas” who have “huge amounts of data and huge amounts of capital,” former Nvidia engineer Jacopo Pantaleoni told The Information.

“That wasn’t the world I wanted to help build,” he said.

Some Silicon Valley veterans have said that the frenetic days of Nvidia GPUs won’t last forever and that other options will inevitably emerge.
Or the cost of entry will prove too high, even for the giants, bringing the current boom back down to earth.