Long Thinking Is Seen As Key to AIs Next Move

"Nvidia's chief executive made a passing reference late in the company's most recent earnings call that deserves more attention than it got -- even if it was understandably overshadowed by $35.1 billion in quarterly revenue, driven 94% higher by customers' ravenous demand for AI chips.

"We're in the beginnings of this generative AI revolution as we all know," Jensen Huang said on the call. 

"And we're at the beginning of a new generation of foundation models that are able to do reasoning and able to do long thinking."

"Long thinking" didn't make it into the zeitgeist when OpenAI's ChatGPT first stunned the world two years ago with rapid replies to questions about almost anything. But it has the potential to reduce or eliminate the errors that frequently peppered those responses.

The idea is just what it sounds like, at least at the highest level: Long-thinking AI models are designed to take more time to "think over" the results they generate for us. They will be intelligent enough to give us updates on their progress and ask us for feedback along the way.

That can mean spending a few more seconds on a problem -- or much, much longer, as Huang indicated in another telling remark last June.

"In many cases, as you know, we're now working on artificial intelligence applications that run for 100 days," he said at the Computex trade show in Taipei.

As the models' reasoning ability develops, AI is expected to evolve far beyond the current tech that works on our behalf in customer service or automation, or the even more sophisticated agents that are just beginning to appear.

OpenAI's long-thinking capabilities advanced in September with the launch of its o-series models, which it said are designed to spend more time thinking before they respond, reasoning through complex tasks and solving "harder problems than previous models in science, coding, and math."

Catherine Brownstein, an assistant professor at Boston Children's Hospital and Harvard Medical School who researches extremely rare diseases, said OpenAI's new reasoning capabilities are accelerating her work.

"I use it frequently to dramatically cut down on the not-so-fun parts of my work, like summarizing yet another study that might or might not be relevant to the question I'm asking," Brownstein said. "I've also been able to make connections I probably wouldn't have been able to do, due to o1's ability to distill complex genetic concepts into accessible explanations."

The idea of long thinking builds on a dichotomy in human thought that the late Daniel Kahneman referred to as System 1 and System 2.

"System 1 operates automatically and quickly, with little or no effort and no sense of voluntary control," the Nobel Prize-winning psychologist wrote in his book "Thinking, Fast and Slow." System 2 "allocates attention to the effortful mental activities that demand it, including complex computations."

You can guess what dominates AI right now.

"The AI that we are currently building is basically like System 1," cognitive scientist Gary Marcus says. The inherent limitations in that approach are part of why Marcus believes society needs AI guardrails to avoid a "Sorcerer's Apprentice-style mess."

Long thinking is an effort to bring AI into System 2.

The reasoning capability of the new models is still in the early stages, but is on track to make significant advancements next year, according to Srinivas Narayanan, vice president of engineering at Open-AI.

"We're going to have AI systems that can talk more fluently with us, that can also visualize the real world," Narayanan said. "And this combination of reasoning and multimodal capabilities, I think, is going to enable us to build more powerful agentic applications next year."

Software-as-a-service pioneer Salesforce continues to ramp up investment in its Atlas Reasoning Engine, the brain behind the AI agents that became generally available in October, according to Silvio Savarese, the company's chief scientist and executive vice president of AI research.

"We're powering agents, and our own Agentforce, toward System 2-type reasoning, enabling AI to deliver deeper insights, drive sophisticated actions, and create meaningful impact across business functions," Savarese said.

The rise of applications built on System 2 models could help drive a return on the massive investment going into AI. Sequoia Capital partner David Cahn says Nvidia infrastructure needs to collectively generate $600 billion in lifetime revenue to justify companies' spending on those systems over the course of just one year -- and it wasn't anywhere near on track to hit that mark soon.

But reasoning models simultaneously stand to boost demand for that AI infrastructure, including chips, software and data centers. They require an increase in what is known as inference, or the kind of computing that trained AI models do when they respond to users' prompts. Inference also is an area where Nvidia platforms shine.

As Nvidia said last month on its call with investors, "Inference compute scales exponentially with long thinking."

In other words, long thinking is part of the long game for the economics of AI." [1]

1. Technology & Business: Long Thinking Is Seen As Key to AIs Next Move. Rosenbush, Steven.  Wall Street Journal, Eastern edition; New York, N.Y.. 12 Dec 2024: B.4.