As AI Matures, Chips Will Get Customized

"Future computer chips may be able to help ease the alarming energy demands of generative artificial intelligence, but chip makers say they need something from AI first: a slowdown in the pace of change.

Graphics processing units have dominated the bulk of training and running large-scale AI models so far. The chips, originally built for gaming graphics, offer a unique blend of high performance with the flexibility and programmability required to keep up with today's constantly shifting swirl of AI models.

Nvidia's dominance in the GPU market has propelled it to a trillion-dollar valuation, but others, including Advanced Micro Devices, also make the chips.

As the industry coalesces around more-standardized model designs, there will be an opportunity to build more custom chips that don't require as much programmability and flexibility, said Lisa Su, chief executive at AMD. That will make them more energy-efficient, smaller and cheaper.

"GPUs right now are the architecture of choice for large language models, because they're very, very efficient for parallel processing, but they give you just a little bit of programmability," Su said. "Do I believe that that's going to be the architecture of choice in five-plus years? I think it will change." What Su expects isn't a shift away from GPUs, but a broadening beyond GPUs.

Nvidia and AMD haven't been vocal around specific plans. Nvidia declined to comment for this article.

Some custom chips are already at work handling aspects of AI. Large cloud providers like Amazon.com and Google have developed their own custom AI chips for internal use, such as Amazon's AWS Trainium and AWS Inferentia, and Google's tensor processing units, or TPUs. These are built to execute specific functions.

Custom chips can be more energy efficient, cheaper and smaller because they can be hard-wired to a given degree: They can perform one specific function, run one specific type of model or even one specific model itself, said Shane Rau, research vice president for computing semiconductors at market intelligence firm International Data Corp.

But the market for commercially selling these super-custom, application-specific chips is still immature, Rau said, a symptom of how much innovation is happening in AI models.

Highly customized chips also present a challenging lack of flexibility and interoperability, said Chirag Dekate, a vice president analyst at research firm Gartner. To the extent they are programmable, they are very difficult to program, typically requiring custom software stacks, and it can be difficult to make them work with other kinds of chips.

Many chip offerings today exist on a continuum, with some GPUs that can be more customized and some specialized chips that provide a level of programmability. That gives chip makers an opportunity, even before generative AI becomes more standardized.

There is no one-size-fits-all when it comes to computing, said Su. AI models in the future will use a combination of different types of chips, including GPUs and more specialized chips still to be developed, for various functions." [1]

1. Business News: As AI Matures, Chips Will Get Customized. Bousquette, Isabelle.  Wall Street Journal, Eastern edition; New York, N.Y.. 23 Sep 2024: B.3.