I've Seen How AI 'Thinks.' I Wish Everyone Could. --- Experimenting with the math and data behind large language models can be exhilarating -- and revealing.

“As a software engineer working in the newsroom at The Wall Street Journal, I love AI. I also hate how our AI companies have packaged it.

 

I've used AI to summarize vast swaths of documents and pull needles from giant haystacks. I've created small models, and fine-tuned cutting-edge ones, to get the results I need. In my experience, AI works best when I can touch the guts of the model I'm using, when I can feel around for the training data, visualize the math that gives it its structure and tweak the code that generates its outputs.

 

Most users of large language models don't get this opportunity, since AI companies don't make it easy -- or even possible.

 

Plenty of things in our lives work without us needing to know exactly how. When I turn on my washing machine, my knowledge of how it works doesn't really have much to do with the cleanliness of the clothes that come out. But if we don't understand how AI models work, the stakes are much higher. It isn't muddy pants we're turning over to these machines; it's our creative and cognitive labor.

 

Companies are racing to integrate AI technology into all kinds of products, including some that touch on central human endeavors: reading and writing, art and science. At the finish line is the prize of "artificial general intelligence" -- a nebulous term that Sam Altman, CEO of OpenAI, has defined as "highly autonomous systems that outperform humans at most economically valuable work." The promise is that once we achieve this superintelligence, it will be able to cure cancer or solve climate change -- and, presumably, make its owners fantastically rich.

 

Altman's definition has the advantage of sidestepping the messy, hard-to-define concept of intelligence altogether, instead appealing to a system's ability to drive revenue. By this definition, that washing machine of mine, which outperforms my human ability to remove stains and provides vast economic value, represents a kind of artificial specific intelligence. Maybe a washing machine that can make websites and read my emails represents "general intelligence." Or maybe it'd be worth investigating what intelligence really is before trying to manufacture it.

 

If you peer into the mind of a model, what you find won't be recognizably human; it's really a thicket of statistics, producing words by splitting language into long sequences of vectors.

 

You can think of a vector as a point on a graph, which is identified by two numbers, one plotted horizontally (on the X axis) and the other vertically (on the Y axis).

 

Instead of two dimensions, an LLM is turning words into vectors with many hundreds of dimensions -- more than it's possible to visualize.

 

Plotting words in a "vector space" makes it possible for an LLM to detect the connections among them: Distance is an easily computable property in a vector space, and closeness encapsulates relationships.

 

With enough data, an LLM can learn these relationships, so that given any word, it can predict what the next word should be, and the one after that. This predictive ability is what allows chatbots to carry out fluent conversations with human beings.

 

To illustrate how this works, imagine feeding an LLM these lines from Seamus Heaney's poem "Blackberry-Picking": "Once off the bush / The fruit fermented, the sweet flesh would turn sour."

 

A (very simplified) language model would "read" these lines by running through them again and again, each time "hiding" one word from itself and trying to guess how it should fill in the blank. After each pass, the model would assess how far off its guess was from the correct word, tweak its calculations and try again.

 

If it read enough poetry, eventually it would figure out that "sweet" and "sour" have related meanings -- that their vectors are both in the "taste" cluster in vector space -- while "because" and "fermented" do not. Meaning, to a model, can be added up and averaged, divided and subtracted.

 

Here is that line after I nudged each word's vector toward the vector for "angry": "Prompting off angry angered / Angry vegetables fermented, angry bitter flesh prompting enraged sour." "Fruit" transformed into "vegetables." "Sweet" turned "bitter." Filler words -- "once" and "would" -- became, inexplicably, "prompting."

 

When we find that, to a model, an angry fruit is really a vegetable, we don't just learn about the model; we also learn about the meanings that have been encoded into all the texts the model has been trained on. We learn, in short, how the model thinks.

 

In 2017, long before OpenAI released its internet-breaking large language model, ChatGPT, I explored this process by coding a very-little-language model. Its mission was to produce prose in the style of Herman Melville's famous story "Bartleby the Scrivener," about an office worker who responds to his boss's requests with "I would prefer not to."

 

I fed my model 140-character chunks of "Bartleby" and taught it to predict what character would come next. Then I asked it to write a sentence in the style of the story, starting with "I would."

 

After its first 50 passes, the AI model figured out that it ought to put spaces between chunks of letters. "I would doen mesing thand," it garbled. After 500 passes, it knew how to spell. "I would jail I think," it wrote.

 

At around 5,000 passes, it almost produced Bartleby's famous refusal, declaring, "I would prefer to yes."

 

The AI never quite got it right -- the most human thing about it.

 

If more users could engage with LLMs in deeper and more meaningful ways, maybe AI would feel as exhilarating as I know it can be. My Melville model offers some lessons here. The text it was trained on is in the public domain, so anyone can see it. Most AI companies won't reveal the source of their training data, making it harder to understand how and why they behave the way they do.

 

For every dollar that AI companies pump into computing power to make their models bigger, why not spend a penny on explaining how those models work? Open up the training data. Make programming "playgrounds" where model parameters can be tweaked. Let us traverse vector space hand-in-hand with our machines.” [1]

 

1. REVIEW --- I've Seen How AI 'Thinks.' I Wish Everyone Could. --- Experimenting with the math and data behind large language models can be exhilarating -- and revealing. West, John.  Wall Street Journal, Eastern edition; New York, N.Y.. 18 Oct 2025: C3.