AI Risks Choking Off New Knowledge

 

“In January, OpenAI Chief Executive Sam Altman and Chief Product Officer Kevin Weil hosted a demonstration of ChatGPT's soon-to-be-released "deep research" application. A Beltway audience watched as Weil asked ChatGPT to prepare a memo briefing a fictional senator for the confirmation of Albert Einstein to be energy secretary.

 

ChatGPT soon produced a thorough profile of Einstein, listing his technical and engineering accomplishments, leadership style, strengths ("a globally respected scientist-statesman") and weaknesses ("never managed a large organization") plus questions the senator could ask ("You have been an outspoken voice on nuclear issues since WWII. As Energy Secretary, how will you ensure the safety of nuclear power plants and uphold U.S. commitments to nuclear nonproliferation?").

 

The benefits of such impressive, and now routine, capabilities, were obvious: enormous savings of time and effort. Of course, there were potential costs: How many jobs could researchers, writers and other knowledge workers lose to artificial intelligence?

 

I wondered about a different cost: How much knowledge will be lost to AI? Large language models (LLMs) such as ChatGPT, Google Gemini and Anthropic's Claude excel at locating, synthesizing and connecting knowledge. They don't add to the stock of knowledge.

 

When humans answer questions, such as whether Einstein should be energy secretary, they often pursue novel avenues of inquiry, creating new knowledge and insight as they go. They do this for a variety of reasons: salary, wealth, fame, tenure, "likes," clicks, curiosity.

 

If LLMs come to dominate the business of answering questions, those incentives shrivel. There is little reward to creating knowledge that then gets pureed in a large language blender.

 

Consider the fate of Stack Overflow, a website where software developers ask and answer questions, becoming both a wellspring and repository for knowledge.

 

But then developers started putting their questions to ChatGPT. Six months after its introduction in November 2022, the number of questions on Stack Overflow had fallen 25% relative to similar Chinese and Russian language sites where ChatGPT wasn't an alternative, according to a study by Johannes Wachs of Corvinus University of Budapest and two co-authors.

 

The drop was the same regardless of quality, based on peer feedback, refuting predictions that AI would displace only low-value research.

 

As of this month, the number of questions is down more than 90%. Why should anyone other than Stack Overflow's owners care? Because, as tech writer Nick Hodges explained in InfoWorld, "Stack Overflow provides much of the knowledge that is embedded in AI coding tools, but the more developers rely on AI coding tools the less likely they will participate in Stack Overflow, the site that produces that knowledge."

 

Many LLMs are trained on Wikipedia, a repository of knowledge compiled and curated by humans. Columbia University business professor Hannah Li and five co-authors found that between the year before and the year after ChatGPT's launch, views fell for Wikipedia pages most similar to what ChatGPT could produce.

 

Meanwhile, as Google has enabled users to answer queries through AI without clicking on links, web publishers have seen referral traffic from search plummet.

 

If LLM output comes to dominate the web, the web will become, well, dumber. Columbia's Li said in an interview: "What happens when we train LLMs on other LLM outputs? The overall outcomes get worse. The models get worse. This is what they call model collapse."

 

There is a parallel in what index funds and other passive strategies have done to the stock market. They don't do research and price discovery (the process of negotiation that reveals an asset's value). Instead, they free ride on the research and price discovery of active investors. In other words, they exploit market efficiency without contributing to it. In the process, they are squeezing out active investing, leaving a market increasingly dominated by algorithms trading against each other.

 

These are, I'll admit, dystopian scenarios. I could tell a different story of how AI will help scholars discover connections between otherwise disparate bits of knowledge across the web. Joshua Gans, a University of Toronto economist who has written extensively on AI, thinks that so long as new knowledge has value, it will find a way to be created. He says when AI insights are incremental, humans will pivot to more truly novel research.

 

Maybe. But instead of pivoting, what if humans lose interest in learning altogether? Reliance on AI can cause critical thinking to atrophy, just as reliance on GPS weakens spatial memory. A study by Nataliya Kosmyna at Massachusetts Institute of Technology and seven co-authors asked three groups of subjects to write essays, one using an LLM, one using internet search, and one just their brains. Scans later showed the LLM group had the least engagement across brain regions such as for memory recall and executive functioning; the brain-only group had the most.

 

Mental engagement, the authors argue, is enhanced by "novelty, encountering new or unexpected content." That resonates. Dissatisfied with OpenAI's demo, I searched the web for biographies and writings of Einstein.

 

I learned that he was outspoken in support for civil rights in the U.S. and against oppression of Jews in Germany, for which the Nazis put a price on his head; that during the McCarthyite fervor of the 1940s and 1950s he was called a foreign-born agitator spreading communism; that he wasn't a communist but was a socialist. In a 1949 essay for a socialist journal, he answered a question I often ponder: how economics differs from the physical sciences: "economic phenomena are often affected by many factors which are very hard to evaluate separately."

 

I have no idea if any of that bore on his qualifications to be energy secretary. I could have spent the time on more productive work. But then, acquiring new knowledge has never felt like work.” [1]

 

1. U.S. News -- Capital Account: AI Risks Choking Off New Knowledge. Ip, Greg.  Wall Street Journal, Eastern edition; New York, N.Y.. 10 Sep 2025: A4.