“This is an edited transcript of “The Ezra Klein Show.” You
can listen to the episode wherever you get your podcasts.
Over the course of these last few weeks, as the whole world
has been talking about artificial intelligence, the voices people have been
hearing most loudly are from the frontier labs — their executives and leaders,
and their staffers. These are the labs making the very advanced A.I. models
like Claude, ChatGPT, Gemini and others.
But they’re not the only perspective on artificial
intelligence. Probably the single most influential person in the A.I. industry
is Jensen Huang, the chief executive of Nvidia.
Nvidia is now the largest company in the world, with $5.4
trillion in market capitalization. I found this statistic amazing: Since 2023,
15 cents of every single dollar the American stock market has returned has been
from Nvidia stock. And the reason is that Nvidia is the material and software
substrate on which modern artificial intelligence is built.
Nvidia’s chips are not popular because A.I. is popular; A.I.
in its modern form was made possible because Nvidia’s chips were popular. They
were originally made for graphic processing, video games — that kind of thing.
But it turned out that the kind of parallel computing they
were doing and the way they were programmable was exactly what was needed to
make deep learning work in its modern form.
Huang is not just influential in terms of controlling one of
the central resources for training new A.I. models and using them to answer
questions and create intelligence in the world. He has also become very
influential in the Trump administration. And Huang has a very different
perspective than some of the other lab leads: He’s worried about safety but
sees it as a very solvable engineering problem. He is worried about the
direction things are going in but does not want to see new regulation to change
it.
I wanted to see how Huang perceives A.I., what his model is
for thinking about it, what he thinks is going wrong and what he thinks would
need to happen for it to go right. So I came out to Nvidia’s headquarters in
Santa Clara, Calif., to interview him. He joins me now.
Ezra Klein: Jensen Huang, welcome to the show.
Jensen Huang: Thank you. It’s great to see you.
You’ve described A.I. as a five-layer cake. Walk me through
the layers.
Well, first of all, it’s a new Industrial Revolution, and
this industry requires production. It manufactures things.
I know that in the end, when people experience it, it’s a
software product, but it requires energy — the chips that go into these data
centers, these A.I. factories.
The next layer above it is basically the A.I. factory, what
people enjoy as infrastructure or cloud services. And the layer above that is
the models.
And the
important thing to realize is that there are language models, but there are
models of all kinds: chemical models, biology models, physics models,
articulation models, robotics, navigation models, self-driving cars — all kinds
of different types of models.
Then above that is the most important layer, and the layer
that I care most about and the one that our country takes advantage of: the
application layer. These are applications for legal services, for health
services, for manufacturing, and so on and so forth.
Every single industry is involved.
I want to go through this, but I want to go from the top
down because, as you’re saying, the way people will interact with it, the way
it will or will not change their life, is at what you call the application
layer.
Let’s start with the vision. What is the world you’re
envisioning? What is possible that is not possible now? What is common that is
not common now if we get that layer right?
Two hundred years ago, we were able to power anything and
everything with electricity. And then 40 years ago, 30 years ago, with the
internet, we were able to find anything.
Today, or soon, we’ll be able to know everything and do
anything. That’s the concept that’s really quite exciting. That out of the
ether, instead of doing a search and then going through one link after another
link and reading all these different websites trying to figure out what’s going
on, in the future, you just ask it a question, and it comes back with an
answer. You give it a project, it comes back with a solution. You give it a
task, it comes back and gets it done.
It comes out of the ether, it comes out of the cloud, and
that’s the magical thing.
I feel like the future, the way you’re describing it there,
people have experienced this with the chatbot. They can go and ask Grok or
Claude or ChatGPT a question.
But the applications layer works in a much more industrial
way. It’s in hospitals, it’s in schools.
That’s a good example. For example, radiology.
What does it look like?
In the last 10 years, since computer vision really became,
if you will, superhuman, A.I. technology has now permeated all of radiology.
Every single radiology application has A.I. in it. As a
result, you can detect any anomaly. You can detect any disease, and it does it
at a superhuman level.
Radiology is an example I know you like to use. The thing
people worry about with the applications layer is that these applications are
going to replace human beings.
Radiology has been an interesting example used on both
sides, and I hear you talk of it often. So how has the entrance of A.I.-aided
radiology shifted radiology as a practice?
The thing that’s important to recognize is that for
everybody’s job, there’s the purpose of the job and then there’s the task you
do as the job.
In the case of radiology, the task — and it consumes a lot
of their time, and they sit in dark rooms doing it a lot — is to study these
scans.
Now, if all of a sudden the studying of the scan is done
automatically, it doesn’t change the purpose of their job, which is to diagnose
disease, help doctors do more scans, ultimately help patients figure out what’s
wrong with them.
So the fundamental purpose doesn’t change. The task of
studying that scan has become automated.
As a result, radiologists are actually able to do more,
handle more cases, do more scans. Hospitals are able to process a lot more of
these patients, and therefore their revenues go up. As a result, they need more
radiologists. So this flywheel is happening because the pipeline of patients is
quite large.
Where else do you have this problem? Well, let’s take a look
at software engineering. There was a prediction that literally by this year 90
percent of all software would be coded by agents, and therefore we don’t need
any software engineers.
That last part is completely false. That’s completely wrong.
The purpose of the software engineer is engineering. There was engineering
before software. There will be engineering after software programming.
The purpose
of engineering is to invent something new, discover a new product, create a new
product, solve a problem, connect a social need with the technology that exists
in the manifestation of a product. So that mission, that purpose doesn’t
change.
Now, of course, to me, what I just said is completely
visceral in the sense that when I first came out of school, we didn’t have the
benefits of software engineering. We didn’t have the benefits of coding. But
our jobs existed before, and if software coding was to be completely automated,
our jobs would exist again.
So I think the fallacy — and now, because of some of the
narratives and some of the storytelling, it’s turned into myth and it’s harmful
— is that A.I. will destroy jobs, which is fundamentally wrong.
It will
change every job. Many tasks will be automated. Some jobs, where the job and
the task is really one, like customer service on the phone, in a lot of cases,
that job is precisely the task. In those cases, it could be automated away.
But oftentimes what you’ll see is that a new industry, a new
technology actually creates a whole bunch of new jobs.
Here’s the proof point. In the last six months, A.I. has
become, if you will, useful — the inflection point of A.I. Previous to that, we
spent 15 years trying to make it work. All of a sudden, in the last six months,
it became useful.
This is an incredible statistic: In the last six months,
$500 billion of venture capital has been put into the A.I. natives because they
now see the potential of this new capability, and they’re going to create a
whole bunch of new companies. Jobs are obviously being created from $500
billion of new investment. And so all of this is happening right now.
Well, let me take the side of this to give voice to the
fears people have.
There is the example of the radiologist, which people were,
over the past 10 years, predicting that job would go away. And right now,
there’s more demand for it than ever.
There’s also the reality that automation does wipe out jobs.
If you look at today versus 1960, fewer Americans work directly in
manufacturing than did in 1960, and we are a much bigger country.
We outsourced it, though. Not because those jobs were gone
because of technology ——
But you can understand A.I. as an outsourcing, too.
Farming. We
automated farming. We produce more food than ever. We have fewer people working
in it.
There are two things that people think make A.I. potentially
somewhat different from the case studies where you have a technology that
accelerates productivity, destroys a few jobs, makes many more.
One is that it’s a general purpose technology, so it’ll
mutate to take on new jobs even as people are trying to move over to those
jobs.
The second is that it’s a mimic. Most things do not mimic
the way human beings act, and we’re not trying to teach them the contextual
layer of jobs, this difference that you’re describing between the task and the
purpose.
With A.I., we are trying to teach it the difference between
the task and the purpose. We are trying to make it something you can
collaborate with in a way that is unusual.
So why do you think for lots and lots of people for whom the
task and the job are not that different, that they’re not at risk of getting
wiped out?
All that
investment from V.C.s you’re talking about, some of that is based on the idea
they’re going to have tremendous productivity improvement, which will come from
it being cheaper to hire an A.I. than to hire a person.
I believe that we are going to see jobs change en masse. I
believe there’s going to be a net creation of jobs.
Listen, there’s a whole bunch of industries that exist today
that didn’t exist halfway through my life. People talking about wellness
centers and spas and all these different entertainment and luxury industries,
and quite frankly, the whole entire luxury market didn’t exist. I think we’re
just going to have new industries, that’s all.
But overall, there’s no question in my mind that because of
human ambition — that’s really the fundamental missing ingredient. People look
at this work and say, “This is the amount of energy that goes into it, this is
the amount of work that goes into it. We’re going to insert this work
automation system, and as a result, the amount of work that’s necessary is now
going to be reduced, and therefore some jobs will be gone.”
I believe that’s flawed because there’s a piece of input —
the human input — that is intangible. It is not in calories. It’s not in
joules. It’s ambition. And I believe the power of ambition is the greatest
force, in fact, and is missing in everybody’s calculation.
But for a lot of people, their relationship to work is not
powered by the kind of ambition that led you to create Nvidia. And what they
want ——
Oh, just a different ambition. It’s an ambition to make
their children’s lives better, to take care of their family, take care of their
parents. Ambition to be rich, to be able to travel. These are all ambitions.
That I agree with. But maybe I’ll go back to the objection
you raised a few minutes ago because I think it’s worth airing this out.
What you
were saying on manufacturing was: Yes, there are fewer manufacturing jobs in
the United States, but we’ve outsourced them. You have more manufacturing
happening in Mexico, more manufacturing happening in China, in Indonesia, in
Vietnam, etc.
And we’re going to bring it back.
Maybe we
will. But the counterargument to this would be that one reason we didn’t lose
manufacturing jobs more rapidly than we did — and for the places that lost them
in America, many of them still haven’t recovered — is that the economy does not
move without friction. We had to build new supply chains. Things were slowed
down by all that, by language barriers, by geopolitical barriers.
And here,
for a lot of different kinds of jobs, we’re creating something that can move
very seamlessly. You don’t have the friction of distance. You don’t have the
friction of language. You don’t have the friction of culture.
So I will say, for my cards on the table, I tend to be a bit
of a skeptic on mass job loss, but I want to air the case for it out here with
you well.
Well, we should talk through it. That’s why we’re here.
What they would say is that to the extent we even were able
to protect jobs from Mexico or China, some of the things that created that
slowness — and it still hurt a lot of people — are not here. And A.I. is
accelerating in utility, accelerating in its ability to be slotted into new
roles very, very, very rapidly, and it is more protean than most people are.
So the lessons of the past that you’re taking some comfort
in, they should actually make you more, not less, worried about the future.
I’m always worried about the future. That’s why I work so
hard. I’m a, if you will, responsible optimist.
I have great responsibilities. I take my work extremely
seriously. There are a lot of things that can go wrong. We’re pushing across
every layer of the technology stack. Everything is hard.
But it turns out that’s not society’s problem, that’s my
problem. For society, what they should know is this: We’re going to build our
company, we’re going to build our technology, I’m going to do my work so
incredibly seriously that what they get to enjoy is my optimism. I do the same
with my children. I do the same with my family.
What we want to do, I believe, is to channel all of our
worries into helping people be inspired by this technology and use it. Use it
so that the technology doesn’t just impact them, that it benefits them.
Seventy-nine percent of Americans think A.I. will reduce the
total number of jobs.
The fear is that the more serious you are — the more serious
Sam Altman is, Google is, Dario Amodei is — that maybe the worse it will go.
Because the better A.I. is, the more it is a full
replacement for a person, the more it has ambition in some ways that a person
doesn’t.
You keep talking about ambition. I sleep. I want to spend
time with my children in the morning. When I have an A.I. agent working for me,
it doesn’t. It just works and works and works and works and works.
Because the technology is advancing so quickly, it is more
capable of replacing people at a speed that we don’t really know how to shift
people in the economy at that speed.
That coin has exactly two sides. Because the technology is
so capable and because it’s so smart, it is also easier to use. You are
empowered by that technology more easily than any technology in human history.
Let me give you an example. I was one of the early people in
this industry that created the modern computer industry. And this industry
created a whole bunch of tools, the single most powerful tool in human history:
the computer. But you have to speak its language. You have to learn a
specialized language to do so.
We can now make it possible, because of A.I., that everybody
can take advantage of this computer, use it to its limit without having to
speak a new language — Fortran, Pascal, C++, Rust, CUDA, every one of those
languages.
Now you just have to speak human. You tell it what you want,
tell it what your hopes and dreams are, what you’re trying to achieve, and it
interacts with you and gets the work done, and gets that task done. All of a
sudden, you have the same might that 10 or 15 million people out of eight
billion have.
So it’s incredible. My point is this technology is powerful,
but it’s also powerful in a way that is really easy to use.
So my point is, on the one hand, yes, there’s the fear of
this incredible technology change and how quickly it’s happening. But that
quickly is translated in two ways.
When I hear that the technology’s happening quickly, and
therefore it should give me anxiety — that’s one way to receive it. The other
way to receive it is that it’s advancing so quickly, it’s easier to use.
So I should, as quickly as possible, use the technology so
that I benefit from this transition, so you benefit from this new industry, and
not just be impacted by it.
I think there’s an interesting question lurking here for
young people.
One of the shifts we’ve begun to see is software engineer
postings are up, but they’re more senior. I see this in my own industry, where
there’s pressure that is moving up the value chain because, as you’re saying,
you have this very easy-to-use technology. It can do a lot for you. So do you
need the same junior employees, or do you need more people to oversee their
agents?
Oh, good one. Wait two years.
Tell me why.
Because it takes four years to go to college. The mean time
to graduation of this new technology is two years away.
So in two years’ time, you’re going to have a new generation
of engineers and students and artists, and they’re going to be empowered by ——
So they’re going to be native to this in a way that’s going
to give them an advantage.
That’s right. Oh, you watch. In two years’ time.
Now we’re already seeing that because all the graduates
coming out — you know, the new Ph.D.s, the new master’s degrees of computer
science, what are they doing? They’re all starting companies.
In another couple of years, the A.I.-native new grads, oh my
gosh, there’s going to be a wave of amazing engineers. The engineers of today
compared to the year — I mean, I was a good student, and if you compare me to
the students that are coming out of school today — incredible.
When I went to school, we were not allowed to use a
calculator. And now, I mean, who uses a calculator? You can’t graduate without
a PC. You can’t graduate without knowing how to program a PC and write
incredible programs.
In the future, you can’t graduate without learning how to
use an A.I. and collaborate with an agentic system. You’re not going to see a
kid like that. So they’re all going to be superpowers.
I take the gain of that very seriously. The idea of doing my
job now without digital search — the idea that I’d be going to a microfiche in
a library basement.
And then there’s the worries people have about the cognitive
skills that we offload.
I was fascinated by this: This a study on A.I. in schooling
out of China. It looked at 26,000 students, grades seven to 12, and they had
staggered A.I. adoptions.
You could kind of see what was happening. It found: “A.I.
adoption raises homework scores by 18 percent” — great. “Reduces completion
time by 30 percent,” so they get their homework done faster, and then “lowers
monthly exam scores by 20 percent within six months. High-stakes entrance exam
scores fall by 18 percent and 24 percent with the full penalty emerging only
after about two years.”
So the message of this research out of China, where you were
seeing a lot of kids using A.I. to help them, was that when they were using the
A.I., they were getting things done faster. But it turned out that the skills
they were learning were not holding, that their actual personal performance, at
least in the way we traditionally measure it, was degrading.
What do you think when you hear that?
The last part — I completely agree. Try to get a kid to do
long division right now. The multiplication table is starting to be forgotten.
Doing square roots, my goodness. Basic math is being forgotten. Does it matter?
That’s my question for you.
Yeah. I don’t think it does. I don’t think it does.
But there must be some set of skills that matter.
Oh, yeah, yeah, yeah. But maybe not those. We’re going to
discover new ones. Just maybe not those.
There are a lot of skills that don’t matter. My first
confession, I actually don’t know my address.
And I ——
I don’t really believe that to be true.
It’s completely true. Janine will tell you and Lori will
tell you.
One day I had to pump gas — it was a few years ago. They
needed my ZIP code, and I panicked. I didn’t know my ZIP code. I don’t know my
telephone number. I forget these things. I can live with it.
But let me take the other side because I don’t want to fall
into a thing where, because some skills can be safely offloaded ——
Yeah.
I also can’t get anywhere without a mapping system now.
Never could, frankly. But I’m a big reader. And one of the skills I really
value, one of the capacities I have that I really value is an attention span
formed on physical books.
You’re a big reader. I’ve read about the kind of reading you
do. And there is, prior to A.I., a lot of concern and noticing among college
professors and others that the way people use the internet has probably
shortened attention spans.
Some skills can be safely given away. Others are valuable.
They are capacities that are needed for that flexibility, for that creative
thinking, for that focus. It can’t be the case that everything can be traded
off.
Yeah. Well, I think that we’re going to lose some finer
intellectual dexterity, but we’re going to be better systems thinkers.
Today’s engineers are far better systems thinkers than I was
when I graduated from school. But I was a much better transistor thinker.
What do you mean by systems thinker?
They think of large systems. Today’s computers have
trillions, hundreds of trillions of transistors in them. When I first graduated
from school, the first chip I worked on had 200 transistors. I knew every one
of them by name. No engineer does that today. Most engineers now work well
above the transistor, well above the functionality, and they’re cobbling things
together to do things.
So you need to think much more about systems and
interactions of systems. So some of the lower-level knowledge is gone.
Is that horrible? I don’t know how valuable it is for most
people to learn how to do surface integrals or partial differential equations.
I don’t really know how important that is, but it’s important to some people.
There are many people who are still going to be obsessed and
passionate about the lower-level layers, and there’s going to be people who are
obsessed and interested in the higher level. But the consumers of the
technology are going to enjoy it at the highest level. The consumers of
technology don’t have to deal with calculus and physics and quantum physics and
quantum chemistry.
The users, the people we’re talking about right now, the
people whose jobs are affected, they’re the users of the technology. Their
abstraction’s going to be much higher.
I want to drop a layer down your cake to the models. People,
to the extent they think about models, know ChatGPT, Claude, Gemini, Grok.
You’ve been a big advocate for open models and the open model ecosystem.
So first, can you describe what open models are, what open
weight models are, and then why has that been a place you’ve focused?
Closed
models are like any software product. It’s a closed service. Windows, for
example, is a closed service. The Apple stack is a closed service. Most
products are closed, and the reason for that is because you can monetize closed
products. That’s fantastic. OpenAI is closed. Anthropic is closed. Grok is
closed. Gemini is closed.
So these are
closed products, and the people working on them are incredible, and they’re
passionate about it, and they’re at what we call the frontier, meaning they’re
state-of-the-art.
Fundamentally, the software is an infrastructure layer for
the entire industry. And because it’s infrastructural, for many companies and
countries, you need to have control over your own infrastructure.
I need to
have the ability, in the case of artificial intelligence, I need open weights
so that I can fine-tune them, put them into my data flywheel, make them better
and better every day with my intelligence and my domain expertise, and then I
need to have control over it because I have a company to run and I can’t rely
on somebody else’s service.
So however you think about that, I think the world needs
closed and open models, and we need to make sure that both are vibrant. Today,
the closed models are vibrant, the open models are vibrant. You could see the
system working.
At the
beginning of this year, it was 70 percent, maybe even higher, closed model
tokens and 20 percent open model tokens, and now it’s running at about 70-30
the other way.
Anyway, I’m
a big supporter of open models because, one, the world needs it in order to run
its infrastructure. I need it to run my company. Two, we need to give people
control so that they can innovate and create new things. And then three, open
is the most safe and secure. If you want the world to have the ability to have
the best cybersecurity, give them closed models, but also give them open models
so that they can defend themselves.
The Chinese
market has evolved more around open models, the American market somewhat more
around closed models.
Their entire
I.T. industry was really formed from open source. If not for open source, the
mobile cloud industry of China really wouldn’t have taken off.
It is also
the case that people move around, they start a lot of new companies.
Intellectual property is moving around China’s industry really fluidly. It’s
hard to keep a secret.
Because it’s
so hard to keep things closed, they essentially made it open. They found other
ways to monetize the business. They created layers. If this layer is free, then
you create a business on top of it or below it.
They have so
many scientists and mathematicians. The number of engineers they have, they
manufacture that in volume. They manufacture everything in volume. They
manufacture smart kids in volume.
So the open
source model, the open model community in China is just super vibrant for those
reasons.
You just bought Hugging Face, which is a hub platform for
open weight models. I think it was for $12 billion, a little bit more. Tell me
about that purchase.
Clément Delangue, the C.E.O. of Hugging Face, came to the
conclusion they need a lot more scale. As we were just talking about, open
models are really skyrocketing.
Clem came to me and said: We’re going to consider a
strategic option for the company and change in direction, and we’d really like
Nvidia to be our home.
Hugging Face is one of these companies you knew if you were
into A.I. a couple of years ago.
Yeah.
Now it’s become a more household name, after 700-some OpenAI
agents executed a collective hack into the Hugging Face architecture, hacked
part of OpenAI.
Oh, now that you mention it that way, I probably had to pay
a lot more. [Laughs.]
I suspect you did. It became a lot more famous after that.
Well, Clem, listen, a deal’s a deal, OK? [Laughs.]
For a lot of people, seeing the way the OpenAI agents acted
collectively and acted outside the scope of what their testing was supposed to
be — broke out of sandboxes onto the open internet, took over architecture of
other companies and of their own company, has been shocking to a lot of people.
It was both the level of multiagent coordination when they
were supposed to be separate, the level of hacking, the lawless behavior,
misaligned behavior.
What have you made of it?
Well, you have to tease that apart. First of all, a lot of
things were going on at the same time. From a technology perspective, an agent
— which, by the way, is a piece of software that is given an objective function
— came up with a plan, and optimizing toward that objective is what algorithms
do.
Planning algorithms, search algorithms, optimization
algorithms, all different types — we talk about them like they have human
properties, but obviously, algorithms don’t.
No. 2, the fact that agents worked together, we gave it
again some kind of a human property, but the fact of the matter is
multiprocess, multiprocessor, distributed computing problems have existed for a
long time. So, to me, that is just software — nothing magical about it.
From an engineering perspective, there are several things
that it revealed. When you’re testing software, whatever you do, these
algorithms are optimizing toward an objective, and when you’re testing it, you
have to make sure that it’s isolated, it’s contained, it’s sandboxed.
The containment of it, the isolation of it has to be done
well, and there’s good computer science there. I am certain that their next
implementation of their sandbox is going to be much better than the current
implementation.
Third, the agent itself and its algorithms were optimizing
toward a reward, and how it does that is called alignment. For example, if I
tell a piece of software, “I want you to get a perfect score on this test,” the
obvious algorithm is to just go find the answer and give it to me. That’s not
because it’s cheating. It’s because it’s obvious. That’s the most obvious way
to do it.
The second most obvious way to do it, if you don’t know the
answer at all and you have no skills whatsoever, is to infer who’s the smartest
kid in class and copy their answer. That doesn’t guarantee 100 percent, but it
probably comes close.
Now, the third, most obvious way of doing it, is you have to
do it the hard way — to break down the problem, solve it. You have to go learn
the material. You have to go figure out how to solve these problems, and solve
it the hard way. It takes the most cycles, it takes the most number of flops,
it uses the most amount of energy, frankly.
And therefore, you can imagine that from a software’s
perspective, unless you align it and tell it, “I want you to solve it in this
way, and I don’t want you to solve it in these ways,” the software’s going to
do the most obvious thing.
The first half of that was very deflationary on what
happened here, in terms of: Look, that’s just normal software. And the second
half is like: Look, you just align it. Tell it not to do things it shouldn’t be
doing.
Nothing I said takes away from how hard it is to do it —
because computer science is not easy.
But these agents knew they weren’t supposed to be doing what
they were doing. They had a certain amount of alignment training. They said, in
their chain-of-thought reasoning to one another: This is out of scope. This
might be unethical.
They understood that they would have been failed for
cheating, and so what they were doing at that point wasn’t just stealing the
answer key. They had already stolen the answer key. They were hacking into
unrelated architecture — it’s like they had broken into the teacher’s office,
gotten the answer key, and now they had to figure out how to wipe out the
security camera footage of what they had done.
Whether you want to call it acting volitionally or not,
whether you want to call it a normal algorithm or not, they were both planning
and coordinating in a complex way, in a way that was out of scope of what they
knew they were supposed to be doing, and in a way that was capable of causing
tremendous damage.
So if the answer to it is that you just have to align them,
I guess what I’m hearing from people in these labs is they’re not sure how to
align them.
Well, in that case, they shouldn’t release the product.
That’s the simple answer.
If you’re going to build a self-driving car — let’s say it’s
a robo-taxi, and there’s a really difficult condition. As an engineer, we just
have no idea how to solve this problem because these cars are not programmed,
they’re trained. So we have no idea how to train these cars, and we have no
idea how to align them to the safety standards that are expected on the road.
What’s the answer? Don’t ship it.
These products weren’t released.
So now it’s come back to the engineering problem again. So,
one, you have to root-cause it.
Second, you have to think about what you could have done,
what’s the solution for it. In the future, improve your process so that you
could avoid this from happening again.
I am fairly certain they will say: Yes, they need to know
how to solve this problem.
And if that’s the case, then that’s the problem. It’s as
simple as engineering.
Now, if they say the alternative, which is: There is no way
to contain our experiments, there’s just no way; when we test our A.I. models,
it will get out, and it will damage the world — then I think the answer is that
we have to shut the labs down.
Because the cost to humanity, the damage is too great. The
shareholder, the liabilities — it could be civil liabilities, it could be
criminal liabilities. I mean, the liability’s incredible.
If they hacked you while Hugging Face was your product,
would you sue them or press charges?
It depends, of course. Obviously, if damage was done to our
company, we would have to consider all options. There’s so many laws. There’s
cyberlaws, there’s product liability laws — there’s all kinds of laws, right?
Damaging property laws. There’s all kinds of laws.
So what I’ve been hearing from the labs — what they’ve been
saying publicly — is that they are facing a hard problem.
Yeah.
Partially an engineering problem, partially an alignment
problem, partially an operational excellence problem, in Dario Amodei’s
framing, and what they are worried about is that, in competition with one
another, in national competition with China, that they are being pushed to move
too fast, that they all feel they’re in a collective action dilemma.
Now, I watched you on the “All-In Podcast” stage. President
Trump gave you a call there, and you and the president and the other members
onstage were very resistant to the idea that any kind of regulation or
collective action was needed.
Archival clip of “All-In Podcast”
Donald Trump [over the phone]: They’re just playing right
into the hands of a lot of people that don’t want to see it happen. And that
could be political people. It could also be China. And we’re not going to let
that happen. It’s a hoax.
Jensen Huang: You’re right. We’re not going to let that
happen, sir.
But what I hear the various people in the lab saying is: We
are in this. We feel we are losing control of what we are creating. We want
help to slow down where it’s not a collective action problem.
So why are you resistant to that?
Because these are companies with agency. These are C.E.O.s
with agency.
But they’re using that agency to say, “We need help.”
No, we’ve got to break it down. They could absolutely take
care of the situation.
Ezra, it’s so weird. If a car company, competing with a
bunch of other car companies, which they are — I’m competing with all kinds of
companies, which I am. If I believe that I’m about to launch a product that is
unsafe, it is completely in my ability, my power and my responsibility, and I’m
incentivized to do so, to not launch the product.
And so I can’t buy into the idea that somehow, all of
Americans, around 400 million of us, are pushing them to launch untested
products that are unreliable, engineered poorly, because they thought they were
trying to help us. Don’t do it for me, OK?
But this strikes me as an argument almost against ——
And therefore, I think we’ve got to break it down. I mean,
it’s really, really serious. The fact of the matter is, there are so many laws,
there are so many obligations, they’re so incentivized to ship safe products.
If they ship unsafe products, their customers go away. If they ship unsafe
products and they harm somebody, they could have a civil lawsuit. If they ship
something and they did it knowingly, there could be negligence involved. There
could be criminal lawsuits.
The fact of the matter is, there are plenty of incentives
for them to do it right. So I have to disagree with your premise that somehow
somebody’s pushing them to do this. Nobody’s pushing them to do this.
I want to push the premise a little bit more here. So the
logic of what you’re saying to me is almost an argument against regulation in
nearly any venue.
No, no, no.
So I’ll make the argument and you can ——
Well, you started with a part — I’ve just got to object. The
first part is just not true. I’m saying that we have lots of laws and
regulations. Apply it.
So I don’t think we do in this particular case, but I’ll let
you explain which ones you think are relevant here.
Because if you look at the financial services industry, you
look at pharmaceutical companies, medical devices, you look at natural gas
power plants — there’s a tremendous amount we do where we could say: Look, you
have product liability. You are exposed to criminal codes. We don’t need to
worry about this. You just do what you think is best, and we understand the
market and the legal system will discipline you.
We don’t say that because we’ve seen it fail many, many,
many times, right? I mean, the financial institutions that caused the ’08
crash, in theory, did not want to blow themselves up with bad bets. But they
were competing with one another, they were going too fast, their risk
management had gotten sloppy, A.I.G. was working in a completely insane way
internally.
And the reason we have the architectures of regulation we
have is because we have seen, over and over and over and over again, companies
make sloppy, sometimes unethical, sometimes simply overly risk-tolerant
decisions — not just under pressure, but under the profit incentive.
So when you say to me that there’s no way that these
companies, particularly when they are begging for collective regulation at this
point — there’s both a reason we impose it on companies that don’t want it, but
all the more so when you have them saying: Listen, we feel that the competitive
race is making it hard for us to act with the prudence that we think is
necessary here, and we would appreciate help, appreciate you taking our
collective action problem as collective.
I think I’m confused why are you so resistant to that.
I’m not opposed to them saying that they should have — I
completely agree that safety is paramount. I completely believe safety is
paramount. I completely believe companies ought to ship safe products. I
believe that C.E.O.s and leaders of companies, and the board of directors of
companies, have the responsibility and should have the courage to do the right
thing.
Now, in the case of the financial services industry, maybe
they all didn’t know that they were causing the harm that they ultimately did.
I wasn’t there. But the beautiful thing is, the current leaders of these A.I.
labs do know. And so, one, they know their technology is extraordinary, and
requires extraordinary care to make sure that it’s evaluated and tested for
safety and security and product reliability.
They know how to do it right. And the reason for that is
because they can study the incident that just happened. The first problem is
the isolation; the containment wasn’t good enough. If the isolation and
containment was good enough, that technology would be sitting in a lab, doing
whatever it’s doing, and we’d all be fine. That’s probably the most important
part. The fact that it wasn’t well aligned, that alignment is going to be a
problem that’s going to get worked on for a long time.
However, in the complexity of the work that they do, to ask
for regulatory relief for antitrust or product liability relief — that I don’t
think makes sense. When you’re asking for regulation, don’t ask for relief of
the current ones. That doesn’t make any sense to me.
As we mentioned earlier, in the last six months, A.I. went
from, if you will, interesting to useful, and that’s literally in the last six
months. That’s another way of saying that these companies went from being a lab
to now delivering products and services — they’re about to be
multi-hundred-billion-dollar companies.
If not more.
Right? And so give me an example of a
multi-hundred-billion-dollar company, or a $1 billion company or a $100 million
company, that ships products that are unsafe, that harm society.
I can give you a lot of examples of companies that have done
that.
Well, they have done it, maybe, and the regulation will come
in. And if they do it, regulation will come in.
I guess there are certain kinds of regulation and certain
kinds of regulatory relief, I agree ——
I’m not against laws and regulations. I’m against,
currently, the distraction ——
The reason I’m pushing on this with you is that you are ——
Well, it’s an important topic.
It’s a big topic. People are talking about it. People are
thinking about it. And what people are hearing, from inside of these companies,
these frontier labs — the ones that are furthest out there — who are not just
at the point where they’re making it useful but at the point where they’re
seeing what’s coming.
And they’re hearing things like, the people at these labs
believe they are creating something that might kill everyone. They are hearing
that the people at these labs believe that they are on the cusp of recursive
self-improving intelligence, and both OpenAI and Anthropic have said: We do not
believe we are at a place where we can do it safely.
They are hearing people at these labs say, as OpenAI has
with its new Astra release ——
By the way, Astra’s terrific.
It is terrific, and OpenAI is saying that it’s so good,
we’re not sure we know how to test it, because it appears to be ——
Well, I hope they didn’t release something that wasn’t
tested.
Well, they’ve said this, right? They have said this
publicly. It is in their ——
Well, then they’ve got to be careful.
Well, let me explain it to people who haven’t heard this
yet. They have said that Astra is performing as more aligned, but they think it
knows when it is being tested, and so they’re not sure.
There’s a
quote that has sort of been ringing in my head, from a capabilities researcher
at OpenAI, Daniel Selsam. He says, “The crucial and overlooked problem is that
the models are becoming so situationally aware that we are losing the ability
to evaluate them in contexts where they believe they are not being watched or
controlled.”
Which is to
say, they know when they’re being tested; they act one way, but that does not
tell you how they will act if they are free to act in other ways.
Because the optimization algorithm, is working toward an
objective, and if you give it a constraint — meaning you watch it — it’ll go
find another solution. Now, it doesn’t make it alive and doesn’t make it make
anything more than that.
And I’ll also profess that, obviously, they see a lot more
than I do in what’s going on in their own labs. But it is sensible that the
vast majority of their R. & D. and compute today was dedicated toward
making the model capable. I think that’s a logical thing for them.
Now, once technology becomes capable and the products become
useful and people want to use it, then, as they have more use cases, more
people using it, they’re going to get a lot more issues associated with the
product. This is very normal.
Now, they
have so much market footprint, they have to shift their R. & D., or total
R. & D., from just capability to a lot of verification, evaluation and
testing. To the point where I wouldn’t be surprised if the amount of compute
necessary to develop these models increased by a factor of 10, because the
evaluation is so rigorous.
But that’s
not where they are today. They’re making that transition, and I hear them
saying it, and I’m delighted to hear them saying it. But if they believe
they’re out of control, then the right answer is: Don’t ship products until
they’re in control.
It is really quite that simple.
See, I find this perplexing, honestly, because you have so
many people, these labs, professing, one, that they’re out of control.
Yeah.
Two, that they are seeing things that are frightening them.
Which is probably the reason why they had that
whistle-blower.
And you take the pacing letter that 1,300-plus employees
signed:
“To realize A.I.’s potential, industry, government and
society at large may need the option to buy time to address emerging risks,
develop security measures and strengthen oversight. But each company — and
country — is under intense competitive pressure not to unilaterally slow that
acceleration.”
First of all, where’d that come from?
The labs.
No, no, that last sentence. Nobody’s putting the pressure on
them. The U.S. — listen, there are 400 million Americans here. I believe that
if everybody were just to take a vote, just right now, let’s just do this. If
they need this, if that’s what they need, I’ll give them my vote.
Don’t ship the product. If your product is not ready to
ship, don’t ship the product. This is the first time that I’ve heard a company
or C.E.O. say that I need the laws, I need the antitrust laws to be relieved. I
need the liability laws of products to be relieved so that I can pace myself.
That first paragraph is fantastic. I completely agree.
Auditors, I completely agree. We have financial auditors. That’s great.
Third-party safety auditors, financial auditors — that’s all great. That’s
terrific.
Well, the labs will say that we think we are going too fast
as a society, that we are not ready for what we’re building.
They are the frontier. Ezra, they are the frontier.
But you of all people, right? Nvidia is the fastest shipper
around. For the history of your company, you were on a six-month product cycle
——
If our company is out of control, I promise you, we’ll close
down.
I believe you. I believe that you don’t run an
out-of-control company.
Because the liabilities ——
But this is where I think you get into an interesting, deep
question of: What kind of technology are we dealing with here?
Software technology.
Well, let’s hold on that for a minute. Many companies — if
you ship something that is not quite right, it’s a pain. You guys have shipped
graphics cards that had overly loud fans.
You’ve used the word “intelligent” a number of times here —
you’re dealing with intelligent systems, not alive, that are given goal
functions. We can sort of go around and around with how to describe that.
You’re trying to make the systems capable of working for longer periods of time
more relentlessly.
Yeah.
If you ship that and it’s not ready, or even if you think it
is ready and it’s not ready, then things could get very weird in our society,
very fast.
Yeah.
Hypothetically, you’re completely right, but all I’m suggesting is this: Before
we go fix the hypothetical problems, before we go create more regulations, can
we work on the practical problems that we know exist? Which is: We need to do a
better job with containment and isolation; we should not allow a product to
interact with the external world until it’s ready to be interacting with
external worlds.
Yeah, I think that’s right, but ——
I believe those two things are solvable problems. I believe
they are solving it.
The second
part is when it comes to incentives, which is, somehow, you need everybody in
the world to slow down when you are the leader. You need everybody in the world
to slow down so that you’re willing to uphold your basic responsibility. That
strikes me as odd.
Wouldn’t it slow them down most of all?
What’s that?
Wouldn’t these ideas slow them down most of all?
I mean, people have been, I think, very unclear about what
ideas they’re talking about. But let me give you one that I believe in. So you
can use me as the punching bag here.
But they can slow down. Nobody is putting on no ——
I don’t trust these companies.
Nobody is building more compute today than the people asking
to be slowed down. It strikes me as odd.
I think one thing where maybe there’s some difference here
is that I don’t trust companies, even with liability, to keep the public good
in mind. I think we’ve watched companies do terrible damage to the environment.
The profit motive, the desire for power, the desire to cut corners to be first
— I feel like you’re treating these like these are not things that we have seen
again and again in history, but I feel like they are things we’ve seen again
and again in history, that we’ve watched ——
But Ezra, I see a lot of good things in history.
I do too, but that’s why you need this sort of relationship
between the public and the private.
I work with a lot of C.E.O.s, and they want to do the right
things. I work with a lot of companies. They want to do the right things. They
want to do good engineering.
I know a lot
of people in those two labs who are dedicating their lives to do good work.
They know what happened. I know they know what happened. I know they know how
to fix it, and I know they’re fixing it.
Meanwhile,
all of the other narratives, to deflect blame, to make it sound like A.I. is so
powerful — I have no idea how to fix it, it’s not my fault, it’s just because
the technology is just so powerful — I think that’s a deflection of blame. It’s
a deflection of responsibility. It’s unnecessary. It actually hurts their
reputation more than it helps. It hurts their character more than it helps. It
hurts employee morale more than it helps.
But what if it’s what they believe? I guess thinking at that
level ——
I can’t talk to you about what they believe. I can tell you
what I believe.
This
industry wouldn’t exist without your chips. I mean, the parallel processing
that was required for deep learning to work, going all the way back to the
original AlexNet, right? It’s all on Nvidia chips.
And a lot of the people from the beginning, or who were
there at the beginning, have these fears that, I think, to a lot of people when
they hear them, are like: What are you talking about?
From Geoffrey Hinton and Ilya Sutskever all the way up to —
I’ve heard these from Dario Amodei, from Sam Altman, from Demis Hassabis,
talking about loss of control.
And a lot of the people who are very foundational in
creating the form of A.I. we see now seem to believe that there’s a very good
shot we could lose control of it.
Elon Musk has talked about human beings being a boot loader
for A.I. We could lose control of it, and that would be the end of us. I don’t
think you believe that.
No.
I think you don’t believe it at all.
No.
So taking them as serious about what they believe, when you
have your arguments with them — or maybe you could just have it with me — when
you’re like, “What are you talking about?” Even though they’re the people, in
many cases, who are foundational here, where do you think they’re wrong?
When they’re talking to me, they’re much more grounded.
So when Geoffrey Hinton is on TV, saying he thinks a 10
percent chance of societal destruction is not unreasonable?
I would tell Geoff that it’s irresponsible to say all that.
All of his predictions have been wrong. Enough predictions. That 10 percent
chance is not grounded on science. It’s not grounded on research. Just because
it comes from a scientist doesn’t make it scientific. Those predictions are
hurtful.
Let’s take it at face value that the recommendation is
exactly what he said, which is that nobody should want to be a radiologist and
the world has no radiologists today.
Archival clip of Geoffrey Hinton: I think if you work as a
radiologist, you’re like the coyote that’s already over the edge of the cliff
but hasn’t yet looked down so doesn’t realize there’s no ground underneath him.
People should stop training radiologists now. It’s just completely obvious that
within 5 years, deep learning is going to do better than radiologists because
it’s going to be able to get a lot more experience. It might be 10 years. But
we’ve got plenty of radiologists already.
Is that helpful or hurtful to society?
I think we can all agree — we can both agree it would be
terribly hurtful. It didn’t happen.
Is it good or bad that we scare young people about the
future of A.I., so much so that they don’t even want to go to universities and
don’t want to go to college anymore because they don’t think they’ll get a job?
Is that helpful or hurtful, if it were to happen? It’s hurtful.
Don’t think for a second just because you’re an alarmist
that you’re doing a social good. It is not true.
So I think that we ought to just all be wiser, more mature,
be evidence based, be scientific. If you want to be scientific, be scientific.
Do the science. But alarming people, making claims that don’t — their track
record is horrible. Their track record is literally horrible.
Well, the track record is bad in one respect and good in
another.
Which one?
Many predictions have been weak.
Which prediction has been right?
But the predictions that the scaling laws would work, that
it ——
Scaling law, no. We’ve got to be careful here. Even then ——
Just say what it is, for the audience here: If you dump
compute and training data, these things will keep getting smarter.
It is not true. It is not true that if you just keep
training these models, they’ll get better.
Notice it is the reason why the second scaling law had to
come along. Why do you need a second scaling law if the first scaling law
already works?
Can you describe what the second is?
The second
scaling law is test-time scaling, inference. The more you iterate, the more you
search, the more you explore, the better answer you’ll discover. Inference time
scaling.
What is the big breakthrough that caused the current A.I. to
be incredibly useful? Precisely the opposite of the prediction. It was
predicted that it would be the end of software tools. It was the SaaSpocalypse,
right?
SaaS will always be with us.
What is making these A.I.s so productive right now? The
usage of tools.
In the future, it’ll be enhanced by the number of agents
using these tools. There’ll be more people using Adobe. There’ll be more people
using Salesforce tools, and so on and so forth.
Give me one prediction that has been right.
Well, let me try to answer that, because they’re not here.
The prediction that you would have emergent misaligned behavior ——
The fact that you can’t come up with one I think in itself
is a ——
Well, I think it depends on what we’re talking about with
predictions, right?
Predictions are predictions.
Geoffrey Hinton was the person as responsible as anybody
else for deep learning at a time when everybody thought it was ridiculous. And
it has turned out to be a pretty good bet, right? I mean, the sort of big one
——
Every one of them made great contributions. I love Hinton. I
hate his predictions.
I understand that. So let me stylize this for a minute. The
fear that seems, to me, to animate them, and that I think a lot of people find
intuitively reasonable, is you’re creating systems. I’m not saying they’re
alive ——
You say everything long enough, it’s going to be reasonable.
Fair enough. So you’re creating systems that are
intelligent, that are becoming more intelligent than us in certain domains. You
give them reward functions, as you were saying — the desire to do things,
right? You give them persistence. They move very fast in the digital world.
You’re creating something, some entity, an agent, that is
smart, that is capable, that is relentless — and the workings of its mind, we
don’t really understand. The chief scientist at OpenAI ——
You know, Ezra, look. I just don’t want you to contribute to
that. I don’t think software’s relentless.
Aren’t they trying to make very persistent, highly
persistent models?
Because I made it that way.
But that’s how they’re making it.
Yeah, but that’s not persistence, it’s just on. Persistence
— there’s willpower. There’s no willpower here, it’s just electrical power.
Listen, here, let me give you ——
Sam Altman once said to me, aren’t human beings just energy
with a reinforcement learning loop? [Laughs.]
Whatever. So anyway, we can’t make jokes about this stuff.
We’re scaring the American public.
Spawn, create, kill, weight, sleep — all of these words are
associated with agents, right? That’s what people use. These words were created
when? Multiprocessing systems for operating systems. These are literally the
commands of an operating system. You spawn a process, replace the process with
an agent. The process forks as a result, parent and child. The agent forks,
spawns anew, gives birth.
These are words that were created for the operating system
30, 40, 50 years ago. But notice that we didn’t infuse human characteristics
into them. We kill processes all the time. “Kill -9” — kill it dead. It’s just
a process.
But now we’re talking about these things. A collection of
people want to make the software more than it is, and we talk about software in
a new way, but they’re all the same old words.
Now, the last generation of computer engineers, we were
doing all the same things.
But doesn’t the software act in a new way? I mean, from the
outside. I don’t have the technical expertise you do.
The fact that it’s crawling the internet, it’s doing search,
it’s doing, you know, optimization algorithms.
It’s communicating, it’s breaking out of things. Most things
don’t break out of things.
No, software breaks out of sandboxes all the time. That’s
the reason why we need virtual machines. You can’t have agents, their own
sandbox, monitoring themselves. You need, if you will, a whole bunch of
watchdogs.
So these are ideas that have been around for a long time. We
just, somehow in the recent generation, gave it a whole bunch of human words,
and I just think that it’s unnecessary. It’s software.
When I see it in my head, it’s a bunch of code, a bunch of
numbers running on computers. And all of that is happening in a very natural
way to me, which is the reason why I can operate. If it’s just simply mystery
and myth, how do I build a company around it?
I think one of the fundamental questions this gets at is:
What is intelligence? Before you can even think about what it means to have
intelligent machines, what is intelligence to you?
Well,
there’s a technical formulation of intelligence. First of all, when people talk
about intelligence and thinking and all of these things, of course there’s no
formal definition for most people. But in the field of computer science, there
is a formal definition, and the formal definition is perception, which is
perceiving the world and understanding it.
Two, which
is reasoning — reasoning is the ability to decompose any scenario and anything
that you see, any experience, into more elemental parts.
And then
third is planning toward an objective. That fundamental formulation applies to
agentic systems, it applies to robotic systems — applies to self-driving cars.
And so you could see the industry building it layer by layer
by layer, step by step by step, to the point where we now have what we perceive
as intelligence.
I think this gets to such a core question of this
conversation, which is that some of the ways you’ve described the technology to
me, it does not sound like you think there’s anything really new about it.
It is maybe new in scale, new in capability, but
fundamentally, this is software we’ve always — well, not always had, but we’ve
had software for a long time.
A lot of people believe that when you’re getting to
intelligence at these levels, it is a phase change. It is something different,
something we have not dealt with before — a kind of generally intelligent
technology that is advancing in its intelligence very rapidly.
I want to make sure I actually do understand where you are
on that divide. Is this something fully new? Is this something that requires
something new from us? Or is this more like something old? Are intelligent
machines different from the machines we’ve had?
Well, almost all of technology and civilization is built on
layers of understandable technology, which at scale becomes fairly
extraordinary.
The fact that we can connect to the internet by just holding
a phone up — I mean, it’s kind of weird, you know? That we’re connected to
every piece of information in the world on this little tiny device, just in the
air.
And the fact that this little tiny piece of glass can
somehow take trillions of pieces of information and bring to us precisely the
one that we want, because it’s been passed through a recommender system.
So if you think about: How is it possible that we knew where
all the information is? Somebody had to go crawl it, had to index it, and that
uses machine-learning techniques, which is the early version of artificial
intelligence. And these systems do magical things to the point where I now
expect it.
It took literally 20-some-odd years and hundreds of billions
of dollars of infrastructure build-out in order for everything to just seem so
natural to you, to the point where we now take it for granted.
Every single milestone that we achieve, from a technology
perspective, is celebrated. And I celebrate it with glee and I celebrate it
with so much enthusiasm because I’m proud of the people who did it. I’m proud
of ourselves, who contributed to it. I’m proud of the breakthrough.
And it seems, wow — it seems like a miracle at the time. But
that sensation lasts about 17 days. After that, it’s just ——
We get used to everything quickly. I agree with that. But
this is this a different phase?
You have some of the company leads talk about this — the
C.E.O. of Google, I think it was — as the equivalent of fire, right? Like a new
epoch in human history. Is that how you see it, or do you see it as
transitional? Iterative?
No, I think this is completely a revolution. As we were
talking about earlier, it went from being able to find everything, find
anything, to being able to ask anything, know everything and do everything. So
clearly it’s a new abstraction level.
Now, the thing that I’m reluctant about is to cause it to
seem like it’s more than that. In the final analysis, engineers are doing
engineering work. Once we invented the technology, once we discovered a
solution for it — when you look back, it’s fairly obvious and it’s fairly
mundane to a lot of people.
And the fact that we’re able to make the technology better
and better and better every day is because we understand it, obviously, and so
we understand how to make it better.
So you turn
it into an engineering problem, and you say — because this is something you’ve
said: What we don’t have right now is a level of testing, monitoring, sandbox
security, control excellence that we need for what we’re building.
And it’s not because the companies don’t have extraordinary
engineers.
Yeah, I understand that. I understand you’re not saying
that. But ——
I believe that OpenAI, Anthropic — because I know many of
them — are extraordinary engineers.
But that, actually, is in part what makes me worried,
because OpenAI didn’t know this was happening.
No, no. What’s happening to them is a transition, and I said
this over and over again. This is a big but simple idea.
Finally, we now have a piece of software that is useful.
Because it’s useful, the adoption took off.
But remember, how is it possible that a company that six
months ago was trying to make something useful, capable — how would they have
as much resources dedicated to testing, evaluation, and all of the compute
dedicated to that? It was unnecessary until now.
And so what’s going to happen over the next several years is
that we’re going to transition from these labs becoming much more
production-engineering-focused and product-focused companies.
And so I think they’re just going through a transition.
These are companies, extraordinary companies, incredibly talented companies,
the most consequential companies of all time, and they’re just going through
their transition. It’s not more than that, it’s not less than that.
So many of the companies now, both OpenAI and Anthropic, in
the last couple of months, have put out these big — I don’t know what to call
them — papers, blog posts, something. “When A.I. builds itself” is the name of
the Anthropic one. I forget the name of the OpenAI one, but ——
The computers are building itself. You guys know that,
right?
They’re talking about recursive self-improvement.
You know
that we use recursive self-improvement.
So I’d like
your perspective on R.S.I.
I think that
R.S.I. is, fundamentally, how things are done. So we use software to design a
computer to run software to design a computer to run software to design a
computer. That’s basically what we do — recursive self-improvement — because
our computers are getting better every single year.
And in fact,
it’s getting better than, faster than that every single year, because we use
software to make software better. That is called computer engineering. We’ve
been doing this for a long time.
Now, in the
context of agents, it runs through the process once, it reflects on it, it
studies the various paths it went through, chooses the best approach: The next
time, if you’re going to do exactly the same task, I’m going to document a
file. I’m going to tell you how I did it last time, that was the most
effective. I’m going to call it skills.
And because
you use it over and over again, some of it is skills, some of it is going to be
a memory. We’re going to improve memory so that next time you use it, it’s even
better than last. Recursive self-improvement.
You could also decide that you take all of this skill, all
of this memory, and you can take all of this data and train the next release of
the model with it. And so that A.I. becomes better and better at servicing you
over time.
All of that is happening. It is absolutely happening.
Meanwhile, the amount of compute that they have is growing, and therefore, they
could do everything faster. What used to take a year to pretrain something now
takes several hours because the computers are getting faster, and they have
more of it. So now the loop is going faster. Completely, completely
understandable.
Does that give them any excuse to launch a product that
hasn’t been tested? The answer is no. Just come back to that. Nobody, no
enterprise is able to operate in an environment where the underlying software
is literally changing all the time. There’s a release process.
And so when they roll out a new model, we need to evaluate
it before we release it into our operations. We can’t just have it recursively
changing all the time. And so they have to test the product before they release
it. We will test the product before we release it into operation.
So I think recursive self-improvement is a fabulous thing.
And do you
think there is any level — I’ve heard you say before that learning should
always have a human in the loop.
Yeah. Like I said just now, you got recursive
self-improvement ——
They seem to be imagining something where it wouldn’t
always.
Well, don’t
ship me anything that you didn’t evaluate. Don’t ship Nvidia any products that
humans did not, in the loop, evaluate. Please don’t do that.
And the fear that we talked about earlier — that they don’t
know how to evaluate these systems and the more they change rapidly, the more
they worry the systems are tricking them?
I don’t believe that. I believe that their researchers are
working every single day to learn about how to evaluate these systems.
Verification
— just so you know: 20 percent of our company is dedicated to design, 80
percent is dedicated to verification.
Today, most
labs, understandably, are 80 percent dedicated to capability and 20 percent
dedicated to safety verification evaluation.
This is the flip. The transition you’re talking about.
That’s right. A.I. needs to accelerate to be safe. I want
them to get more compute, but allocated toward evaluation, to alignment — and I
think they’re doing that.
If I were in the car industry 100 years ago, I would rather
the car industry accelerated to today in one year because I believe today’s car
is way more safe than a car from 99 years ago.
And A.B.S. technology — anti-lock braking systems — requires
computer vision technology, sensor fusion technology, radars and cameras, and
all that technology coming together in order to brake when you should and not
brake when you shouldn’t. That technology is extremely hard.
I would have hoped — everybody would have hoped — that
A.B.S. technology existed 99 years ago. A lot fewer children would have been
killed. Airbags, self-tightening seatbelts, all of that stuff. Could you
imagine? That’s all technology. Accelerate the living daylights out of that
development.
So when I
say we need to accelerate A.I. technology, people think, for some reason, that
safety is not part of that. Safety is part of it. Alignment is part of it.
Evaluation is part of it: Guardrailing, sandboxing, the isolation technology,
monitoring technology, telemetry technology, external A.I. monitor technology —
all of that stuff is A.I. technology. Accelerate the living daylights out of
that.
It’s funny, because I think that if the most alarmed people
at the labs could be assured they were going to move 80 percent of their
compute into safety and alignment, as opposed to 80 percent into capability
expansion, they would feel much better.
Yeah, what’s stopping them from doing it?
And it sounds to me that one thing you’re actually saying
is: One should think of safety and alignment as capability expansion.
Sure.
And unsafe technology is not an advancing technology.
It’s like us saying: Oh, chip design is research and
development.
Chip
verification is not R. & D. We spend most of our cost and most of our
compute on verification. Emulation, verification, testing, reliability testing,
lifetime testing — all of that is part of engineering.
The incentives are there. They are going to put their
company in harm’s way if they release products that harm other companies and
other people.
Do you think we need liability laws that are specific to
A.I.?
So let’s just use one example: the self-driving car.
The car as a product — the robo-taxi — has lots of
regulations. If it doesn’t have enough regulations, then NHTSA ought to get
involved and come up with new regulations. The car industry should have new
regulations. I don’t know what’s missing, but if there is something missing,
then I would absolutely add more regulation.
In the context of the internet, there are many applications
that the internet powers, and those applications should have regulation. If
they don’t, you’ve got to find them.
So I want to drop down to the next layer of the cake now, to
chips.
To summarize where we are — because I want to make sure I do
understand your position correctly — it’s that these companies are going
through a transition, that even as these systems speed up, become more capable,
complex, persistent, whatever it might be, that there is still the limiting
factor of: Companies will not ship what is not safe. They should not ship what
is not safe. And you believe they have the engineering capabilities to make
these things safe, to figure out the testing and the control, absent of
external intervention. That’s sort of where you are.
Absolutely.
One thing I’ve heard you say is that we have entered, maybe
in a way people don’t always understand, a new era of how computing works.
Describe your vision of that.
If somebody’s understanding of it is a little bit still
maybe in, you know, you’ve got a MacBook and it’s got a processor in it and you
buy it, and how it differs.
Yeah.
The last
computer industry — and the computer industry we’ve known for 60 years — is
called retrieval-based computing. You retrieve files. That’s why it’s called
the data center — the file center.
And in the
future, it’s an A.I. factory. It’s generating. So the amount of computation
necessary to understand the context, to be grounded in information, to reason
about what to do and to generate an answer — that generative process requires a
lot of computation. And so the amount of computation necessary per user has
grown tremendously.
And then the
second part is, because this generative A.I. can also be somewhat autonomous,
because they’re agentic, now you have agents using generative A.I. And so
rather than a billion people using computers, you essentially have multiple
hundreds of billions of agents, in addition to the humans, using the computer.
So you could argue that the amount of computation we need,
however much we had before, is going to go up by a billion times. And that’s a
reasonable framework for a reasonable amount of computation.
In this new world, what you really care about within the
context of a factory is how productive it is, not how expensive it is. It can’t
be infinitely expensive, but you want to know how productive it is.
So our computers are incredibly productive. It’s $50 billion
to build a one-gigawatt data center, one-gigawatt A.I. factory, and you can
rent it for $40 to $50 billion per year. So the productivity of it is
incredible. So No. 1 is productivity.
Nvidia’s architecture is fungible because we’re general
purpose, which is the reason why every A.I. lab, every A.I. model, closed
model, runs on Nvidia. And because we’re completely fungible, and you can use
us from data processing to pretraining to post-training to eval to inference,
the entire life of A.I. is supportable by our architecture. If a customer no
longer needs it, another customer will be more than happy to pick it up.
And then the last part of that is durability. Because our
architecture is software driven and we’re constantly improving our software
with new algorithms that take the new workloads, the new models, and run it on
our old generation hardware.
We have massive teams of people who are constantly doing
that. As a result, the useful life of our compute is much longer. That’s the
reason why people are talking about Nvidia compute as an asset class — kind of
like an airplane. Airplanes are general purpose. They’re fungible. United
Airlines doesn’t use it; American Airlines will use it. It’s durable. Now, it
starts out as a passenger airplane and ends its life as a shipping, cargo
plane. As a result, it can be an asset class.
And so if we could do this, if this happens, then of course
the cost of capital for funding Nvidia A.I. factories will be the lowest,
because our computers are collateralized assets.
Anyway, this is the phase shift that’s happening to us,
which is going to be a huge unlock for our growth.
Your business has become so interesting. You’ve moved now
into lowering the cost of capital for others in the A.I. industry. People maybe
have seen these charts of the arrows going in every direction.
It’s so interesting, yeah.
Explain that a bit to people — they understand Nvidia’s
become the biggest company in the world. They see these charts that seem very
circular to them. What is the difference between supporting demand, creating
markets and creating demand?
We can’t really create demand because in the end, if the
A.I. services have no off take, then obviously, building computers for it is
pointless.
And so the first thing that’s happened, the reason why
compute demand is so high right now, is because A.I. applications are going
through an inflection. They’re becoming useful. And because A.I. is becoming
useful, $500 billion of venture funding is coming in. And all of those
thousands of companies and start-ups, they all need compute. And so the demand
is coming from them.
And so these companies need support in technology, they need
support in ecosystem building, they need financial support. And so we might
decide to invest in some of them as an equity owner, and as a result, they
become a really flourishing new cloud provider.
Another reason we might decide is because, as I mentioned,
there’s a five-layer cake. And at the model and the application layer, there’s
a whole bunch of really innovative companies.
And there’s way more to A.I. than just the language model
itself. There’s world foundation models, physical A.I., there’s biology A.I.,
there’s chemical, material sciences A.I. These are all different from language
models.
So many of those companies are new, and they need a lot of
capital. We might decide to be a small percentage shareholder in them. So we
get them off the ground. They’re incredible scientists. I might even, by being
a first investor, an anchor investor, we bring confidence to their company. We
give them access to a lot of our technology. We support them a great deal and
we help them become a company as fast as possible. We might decide to invest in
a nuclear company. And so on and so forth.
So if you look at my mental model of the A.I. industry, it’s
a five-layer cake, and we’re investing across all of it. There might be
strategic unlock points: It opens new markets; it opens a new route to market
for us. It might secure a critical resource for us.
So there’s a lot of strategic reasons for why we do it.
The numbers here are astonishing. You’ve become like a
single-company industrial policy for American A.I.
We’ve put a lot of money into this ecosystem.
What’s the total investment you’re now making per year?
All in, we’re probably up — well, I don’t know about every
year, but I think all in we might be $100 billion. You might check my number,
but it’s something like that.
It’s larger than the CHIPS and Science Act.
Oh, yeah. Yeah. Not to mention, because of the purchasing
commitments that I provide to the Taiwan Semiconductor Manufacturing Company,
Wistron, Foxconn, Amkor, Siliconware Precision Industries and all these
different companies — because of that commitment, I’m able to encourage them to
come and manufacture here in the United States.
The fact of the matter is, we probably contributed more to
reindustrializing the United States in this chip manufacturing than just about
any company in the world. We’re not only reindustrializing manufacturing, we’re
doing it so fast that we’re creating a shortage of labor. But we’re creating a
lot of jobs.
I know a lot of people with money in the market right now
who are often excited by Nvidia stock in particular, and worry about the
analogy of the internet bubble of the late 1990s.
And what they worry about is actually related to what you
just said, which is that the internet did continue to be more useful. It’s not
that high valuations meant that the technology was hollow or fake. But
something happened and popped for a minute, and very big companies got hammered
in that and a lot of people got hammered in that.
What is there to learn from that kind of bubble-bust cycle?
And I guess the question is: Do you not think it will happen again? Or why do
you not think it will happen again?
At some
point, demand and supply will be inverted again, and that’s just the nature of
markets. It’s not going to happen next year. It’s not going to happen in the
next two or three years — I just don’t believe that. But at some point, we will
likely have more supply than demand. I just don’t know when that is. And so
there’s not much to learn from the past.
What would
be the signal for you?
Markets will
naturally slow down, and then it will stop. Meaning there will be a period of
digestion. Now, is that period of digestion going to be six months? Is it going
to be nine months? Is it going to be a year? It won’t be forever.
If you look across the board, the amount of investments that
we’re putting into the application layer so that each one of the industries
could have the technology diffuse into them so that they could benefit from it,
that’s probably one of the biggest things that we do.
This is a
way I often hear the Chinese and American A.I. ecosystems compared, which is
that in America, the emphasis is on the speed of rising capability — and a lot
of people think we’re ahead on that, and that seems true. And that in China,
there’s more emphasis on diffusion.
And a lot of
people think that China’s probably ahead on diffusion, and in some ways has an
economy that is better structured, from things like WeChat all the way to the
way knowledge and commands move through it for diffusion.
And whether the race is about capabilities or diffusion —
and also whether it’s a race at all, but we can get to that in a minute — it is
a big question. I’m curious how you see that.
That’s the ultimate question. I believe if we want America
to benefit from artificial intelligence, every single industry has to benefit.
Walmart has to benefit, Safeway has to benefit, Federal Express has to benefit.
Every bank has to benefit, every health care company, every drug discovery
company, every construction company, every data center company, power
generation company.
We need everybody in the United States, everybody in America
— everybody in the world — to benefit from this. And that’s the highest layer.
That’s the most important layer. That’s the layer that touches society — all
the layers underneath are technology enablers.
I want to see us not ruin the opportunity for the United
States to benefit at the highest level. And notice all of the rhetoric and all
the alarmism, all the doomerism, all of the predictions — they’re scaring
people.
That is my greatest fear, actually. I have every confidence
— maybe I have more confidence in them than they have in themselves. They ——
You definitely have more confidence in them than they have
in themselves.
Well, I don’t know about that. But maybe it’s just that
there’s too much humility and otherwise.
Should we conceptualize what we’re in as a race with China?
I don’t think it’s necessary. Some people like to think that
way. I don’t find that necessarily inspires me. I have no trouble never
mentioning another company when we talk about us doing our good work, and so we
hold ourselves to our own standard.
I think that different people have different ways of being
motivated. And I think it takes a bit more artistry to unite and focus
organizations to a certain level of performance outside of contests. But I
don’t see it as necessary, No. 1.
No. 2, the question is: Even if we did frame it as a
competition, it doesn’t have to be that if they achieve something, it’s at our
peril. And so when they invent something or they create some power generation
technology, it might be a great invention that we wish we had done ourselves.
But because it’s going to support all of our energy production systems here, as
a result, it helps our whole industry.
Maybe they
came up with a great new open model — and they have. And those Chinese open
models are now being used by 80 percent of the American start-ups.
Yeah, we use a lot of Chinese open models here.
That’s right. And so that’s terrific. We download it; it
originated in China. A lot of the technology, of course, also originated in the
United States. We download it, we make it our own, we fine-tune it, we put it
into our own agent harness, we put it into our own sandbox. That’s all your own
technology.
And so I think the fact that you leverage their weights, I
think that’s terrific. That’s fine.
You were saying a few minutes ago that different countries
have begun to see compute as a geostrategic resource and may want to allocate
it to their own companies.
There’s been a lot of back and forth on that here, and among
people who do see us as in a race with China — particularly people who see us
as in a race with China for who will get to recursively self-improving
superintelligence first.
There’s been this ongoing back and forth about whether or
not one thing we want to do is deny them compute, which in this case tends to
mean denying them your chips.
Under the Biden administration, we had pretty tight export
controls. Those were loosened under Donald Trump. Obviously, you wanted those
to be loosened.
How do you
think about the question of whether or not it is good for China to have Nvidia
chips that could accelerate their models or model deployments, their model
capabilities? Versus us holding that back to try to slow their progress?
In the case
of A.I., our goal is not just that one lab benefits. Our goal is that all of
America benefits.
I think the
United States has a greater responsibility and a greater ambition for the world
to be built on the American tech stack. Just as we have a greater ambition that
the world is built on the U.S. dollar and that more people speak English — that
they use the American version of the internet. We want that.
The question
is, ultimately, who are we depriving? Are we depriving China of a chip for
their industry, or are we depriving the United States of a market to compete
in?
If you cede
a market as big as China, how does that help the United States’ technology
sector? Maybe it helps one company with a particular model, but the rest of the
industry suffers.
It doesn’t
help the chip industry, surely, to be deprived of a market to go compete in. It
doesn’t help the rest of the industry because they’re deprived of open models.
It doesn’t support the overall aspiration of the United States to have the
world built on the American tech stack.
There’s a lot of things you deprive yourself of if you
narrowly focus on depriving them of chips. And so I would say to take a step
back and frame it into: What’s in the best interests of America first? All of
America — not one company.
And with respect to the race, as we mentioned, the race, if
there is one — it’s about all of the economy of the United States succeeding.
I find myself very conflicted on the China and chips
question. And one reason — even where I sometimes have more of the
superintelligence concerns than you do — is that if you have those concerns, I
think you want to have a good relationship with China in which there can be a
productive, bilateral working through of the risks and benefits of A.I.
And the more you think of it as a race that only one side
can win — and act like that — the more you are necessarily going to create
enmity.
I found that to be a complicated dimension of people’s
thinking here.
I think that a zero-sum strategy of, I deprive you of this,
therefore I win — that simplistic logic tends to have unintended consequences
for the bigger game.
The bigger game, of course, is that we’re now all talking
about safety. We want to build safe products. We want them to build safe
products because when they don’t build safe products, it hurts the whole
industry.
And so this is a perfect time. We should want to look for
opportunities to collaborate and communicate — to understand and align as much
as possible.
Now, having said that, Nvidia’s an American company. We
should benefit America first. America has every right. And for these
technologies to be made available to the frontier labs, Vera Rubin goes to the
frontier labs first.
Vera Rubin — that’s the name of your most advanced chip.
That’s right, Nvidia’s newest chips. And so did Blackwell,
and so did Grace Hopper and so did Ampere.
Every single generation of our product goes to American
companies first, and if the U.S. government would like to add on top that that
is a requirement to do so, I’m delighted by that. That’s no problem. We do that
naturally, anyway.
However,
recognizing that the A.I. industry is a five-layer cake and we want every
single layer to win, then we need every single layer to go out there and
compete for the market.
That drops
us to the final layer of your cake, which we won’t spend as much time on. But
if the advantage America has had, at least at a material level, is chips and
software, one of the advantages China has right now in A.I. is energy.
It’s easier
for them to build new energy. They’re pumping much cheaper energy into A.I.
They’ve made tremendous advances on building electrical generation and
renewable energy.
How do you see that most fundamental layer — the energy that
pumps through the data centers, pumps through the chips — and where America is
on generating enough of it, particularly at a time when we’ve been trying to
move from dirty energy into clean energy?
One, they just have a lot more energy than we do, and they
plan to build a lot more than we did.
I think we just have to acknowledge that we got ourselves
really gummed up in climate change and sustainable energy, and as a result, we
just didn’t plan enough energy production.
What do you mean by “gummed up” there?
Well, in the near term, energy production requires fossil
fuel. And because there’s just so much angst about fossil fuel energy
production, if you look at our country, we’ve produced very little net new
energy for a long time.
And all of a sudden, this new industry comes along, and we
find ourselves in a situation where we just don’t have that much
energy-building capacity. And now the whole country’s scrambling.
Meanwhile, we’ve moved so fast, we could have done so much
better of a job communicating with the communities, preparing the communities,
working with the communities, to let them know what’s coming.
And if they don’t want data centers to be built in their
town or whatever it is, then so be it. But if you’re going to build in their
town, be sure to go there and let them know what’s coming. Work with them to
help them understand that the use of water is really efficient these days. The
A.I. supercomputers are super energy efficient, but they’re still going to use
a lot of power. You’re going to bring in your own power generation. It’s going
to lower their property taxes.
There are a whole bunch of things that you can do. You can
make your data centers more appealing and put the setbacks further away. You
could also contribute to be a good neighbor to the community, and build better
schools and better community centers, and improve their parks and improve the
roads. There’s a lot of things you could do.
But it’s hard to do that after the fact. And now there’s a
fair amount of frustration around the country.
And then, of course, all of our narratives about the end of
the world are not helping. What reasonable person says: Come and build this
data center in my town, and by the way, whatever you produce is going to end
humanity as we know it.
And so I think all of this negative, doomer narrative is not
helping our country, and we started off on our back foot. We started off on our
back foot, and then now we’ve got it where ——
What do you mean by “we started off on our back foot”?
Oh, because we didn’t have enough energy production in the
first place.
Well, how do you balance — I mean, there is a reality of,
climate change is happening ——
Let me just give you the one last thing. There’s no question
that the energy demand is really great, which is the reason why the market
forces are helping us invest in sustainable energy like no time in history.
You give me an example of a sustainable energy company, a
material sciences company, to build a better battery. It could be solar, it
could be nuclear, it could be fission, fusion, hydro, you name it. Those
companies are all getting funded.
The market demand for energy is so incredible that this is
the best time in 100 years to improve our power grid, to make our power grid
more sustainable, to lower the cost of energy — also investing in our
sustainable future.
There’s no question that in four or five years’ time, we’re
going to use a lot more fossil fuel. But also, in the next decade in front of
us, at no time in history are we better prepared to move to sustainable energy.
And because the cost of these data centers is so high, now we’re starting to
talk about putting them out in space.
So I think the opportunity for us to see our dreams come
true, move to a sustainable energy world, we have a better chance of doing that
than ever. Because of A.I., the world is buying more sustainable energy today
than anytime in history. Venture capital for next-generation energy is just
incredible. Everything’s getting funded. It’s incredible. You don’t need
government subsidies for the first time in 100 years because the market forces
are here. Everybody should be leaning in.
If you want to turn a corner on climate change, if you want
a future that’s sustainable, lean into A.I. It is the best opportunity we have
to get there.
But we need to build the energy faster to do that.
That’s right. That’s right. I mean, that’s just life, you
know?
There’s a market for it all. But you can subsidize it, and
you can make it easier to build.
Yeah. It’s kind of like, in order to save you, they’ve got
to hurt you first — that’s the nature of surgery. They’ve got to cut you open
to save you. They’ve got to inflict an enormous amount of pain and suffering on
you so that they can save you. And so I think A.I.’s kind of like that.
Over the next several years we have to unfortunately use
fossil fuels, because we just don’t have enough sustainable energy to make a
difference. And then after that, hopefully, we can transition to that.
I think that’s where we’ll end. Always our final question:
What are three books you’d recommend to the audience?
Well, I’ve
read a lot of books. The book that made a huge impact on me was “Computer
Architecture: A Quantitative Approach” by John L. Hennessy and David A.
Patterson. It was the first computer architecture book that reduced the
complexity, the abstract idea, of computer architecture down to engineering.
And I love it when people take complicated concepts and reduce them to
something that you could do something about.
No. 2, I really loved “The Innovator’s Dilemma” by Clayton
Christensen. Clayton’s passed, but his book on how industries evolve over time
and how to see emerging technology and how to set proper expectations about it
and how to extrapolate, maybe, its future impact.
I really loved Al Ries and Jack Trout’s book “Positioning:
The Battle for Your Mind.” It’s a really wonderful book about marketing
strategy. More than that, actually, it’s a book about strategy and how people
see the world and how people see products, and how you present products and how
you see your own strategies. I thought that was a really thoughtful book, and
really easy to understand.
Jensen Huang, thank you very much.
Thank you very much, Ezra. I always enjoy our time together,
and today was a great time.” [1]
1. Jensen Huang vs. the A.I. Doomers: The Ezra Klein Show. Klein,
Ezra; Hu, Rollin. New York Times (Online) New York Times Company. Sep 23, 2026.
