Nvidia Adapted Its Chips To Keep AI Lead

 

"Nvidia faced a growing threat early last year: The artificial-intelligence world was shifting in a way that invited competition.

As millions of people started using AI tools, operating the underlying models to respond to their many queries became more important than the computing-intensive work of training them -- which propelled Nvidia to the top of the AI boom. Many expected that shift could give competitors, including Advanced Micro Devices, an opening to pry away market share.

But Nvidia was already preparing to adapt and stay at the forefront of the AI race despite the shift away from creating models and toward operating them, a process known in the industry as "inference."

Its latest AI chips, called Blackwell, are larger in size, have more computer memory and use less-precise numbers in AI computations. They can be linked together in large numbers with superfast networking, which Dylan Patel, the founder of industry research firm SemiAnalysis, said led to "breakthrough gains" in inference. "Nvidia's performance gains for Blackwell are much larger in inference than they are in training," he said.

Nvidia's quarterly earnings report on Wednesday partly reflected its success in adapting to the industry's shift. It included sales and profit that exceeded analysts' expectations, coupled with an optimistic forecast for the company's current quarter.

Despite the strong results, Nvidia shares fell 8.5% to $120.15 on Thursday, its largest post-earnings percentage drop since November 2018. Investors got cold feet as analysts pointed to narrower profit margins in the quarter and concerns about chip sales in China.

Inference has become a growing focus as AI evolves toward so-called reasoning models, where a digital brain thinks through answers to users' queries step by step. That process can require a hundred times more computing power, Chief Executive Jensen Huang said on a call with analysts Wednesday.

"The vast majority of our compute today is actually inference, and Blackwell takes all of that to a new level," he said. "We designed Blackwell with the idea of reasoning models in mind."

Colette Kress, Nvidia's chief financial officer, added that many early deployments of the company's Blackwell chips were earmarked for inference work. That pattern was a first for a new generation of the company's chips, she said.

Among the companies pursuing reasoning models are OpenAI, Google and the Chinese AI upstart DeepSeek.

The emergence in January of DeepSeek, which said it built sophisticated AI models that required fewer Nvidia chips, touched off the first significant scare for Nvidia since the AI boom began.

Huang brushed off that threat on Wednesday, describing DeepSeek's advances as "an excellent innovation" that AI developers everywhere were taking inspiration from.

In the past, Huang suggested inference and training will converge as AI more closely aligns with how humans operate. People don't absorb new information and reference it separately, he said at Stanford University last year. "You're learning and inferencing all the time," he said.

Nvidia faces strong competition in inference, industry insiders said.

While Nvidia's advances in hardware and investments in its AI software have kept customers around, a variety of new chips from startups and more established chip makers mean it won't be easy for Nvidia to maintain its top position.

Robert Wachen, a co-founder of AI chip startup Etched, which aims to compete with Nvidia in inference by making purpose-built chips, said there was already serious adoption and consideration of alternatives. He said Nvidia's chips were fundamentally limited by their origins as graphics-processing units adapted for AI instead of custom-made for the moment. "Sharpening the Swiss Army knife only gets you so far," Wachen said. "You have to build specialized hardware if you want to get maximal performance. You're hitting a wall here."

A number of startups have begun making inroads among large AI customers. Cerebras, a startup that designs the largest chips ever produced, said this month that it was working with the French AI developer Mistral on the world's fastest AI chatbot. Saudi Arabia's oil giant Aramco is working closely with AI-chip startups Groq and SambaNova Systems to set up large computing facilities for inference.

Nvidia's more-established competitors have efforts of their own, including Advanced Micro Devices, whose AI chips are largely aimed at the inference market. And all of the largest tech companies are developing their own AI inference chips.

Jim Piazza, an executive at information-technology management company Ensono who worked on computing infrastructure at Meta Platforms, said Nvidia might need to take further steps to address the competition in inference by developing chips specifically for it.

"I have to imagine Nvidia is going to drop some kind of inference powerhouse sooner rather than later because I think they will get eclipsed in that market," Piazza said.

Huang is already thinking through a future that involves a lot more computing power -- Nvidia's, he hopes.

Reasoning models, he said Wednesday, could eventually require thousands or millions of times more computing power than their predecessors. "This is just the beginning," he said." [1]

1. Nvidia Adapted Its Chips To Keep AI Lead. Fitch, Asa.  Wall Street Journal, Eastern edition; New York, N.Y.. 28 Feb 2025: A1.