Efficient To a Fault.

"Optimal Illusions

By Coco Krumme

(Riverhead, 256 pages, $29)

The logistics business is a creature of mathematics. Algorithms dictate the most efficient plan for unloading boxes from a container ship, stacking them in a freight yard, managing the truck fleets that transport them to distribution centers and delivering their contents to millions of households. Yet when international trade in goods surged during the Covid-19 pandemic, all those algorithms failed to keep supply chains from becoming seriously tangled. Their dictates, it seems, were no longer optimal.

In "Optimal Illusions," mathematician Coco Krumme explores why the quest for the best so often falls short. The hidden assumption behind optimization, she notes, is that all relevant variables can be identified and accounted for in order to achieve the ideal outcome. Yet unknowns and uncertainties can upend the most carefully laid plans. "Optimization was invented, over time, with the idea of making a system, or a society, more perfect," she writes. But "we often see the opposite happening."

Optimization, Ms. Krumme asserts, has arisen from three shifts in how humans conceive of the world. The first shift is an increasingly atomized understanding that leads us to see individuals as assemblages of tiny bits of data. The second is a tendency toward greater abstraction, especially in computer modeling. The third is automation, which changes scale, devaluing the local and the particular. Over time, Ms. Krumme contends, these shifts have led us to understand reality in new ways -- and to believe that we can perfect it.

The agronomist Norman Borlaug (1914-2009), a plant-breeding wizard hailed for innovations that led to the "Green Revolution," offers an improbable example. Borlaug's quest for optimal plant varieties boosted output per acre and drastically reduced the number of people facing starvation. Eventually, however, it had unforeseen consequences, such as the displacement of diversified family farms by bigger operations growing a single crop with large doses of fertilizers and herbicides.

"The deification of efficiency cast aside traditional methods of letting soil rest and regenerate," Ms. Krumme writes. "Its focus on increasing yields divorced agriculture from farmers, and farmers from land and place." Rural communities eroded. Small processors died out. And the surviving farmers, while more prosperous most of the time, became more vulnerable to price swings because their incomes depended on the value of a single crop.

Optimization, Ms. Krumme says, also lies behind the breakdown of the electric grid in Texas during the frigid winter of 2021. The grid sits atop a mountain of algorithms that project power demand, forecast the output of individual generating plants, and allocate energy based on the prices customers and suppliers are willing to pay and receive. "It's a system optimized to work fantastically, when everything's working fantastically," she observes.

But optimizing investment had squeezed out excess generating and transmission capacity, while the models hadn't accounted for a prolonged spell of sub-freezing temperatures. When customers turned up the heat, the electricity they counted on was to be had only at extortionate prices, if at all. What was optimal in certain circumstances turned out to be suboptimal when the weather changed.

What's the problem? Optimization is a mathematical procedure purported to be rigorously objective and free of biases. The basic concepts are taught in every calculus class. A simple computer program can do the heavy lifting. "We start off with what we imagine is a blank slate," Ms. Krumme writes. "We throw in piles of data and let the computer code spit out the 'best.' "

But the mathematical analysis is often divorced from reality. Constraints may be ignored, misunderstood or forgotten. The specific knowledge possessed by individuals who know the idiosyncrasies of a machine or a market is disregarded. And the slate on which a model is built may not be blank at all. While genetically modified crops may be optimal in terms of tons per acre of farmland, how does one weigh the loss to growers who wish to continue planting traditional varieties but cannot do so when the local farm co-op no longer wants to purchase and store what they've grown?

Moreover, optimization cannot readily be undone. The Colorado River Compact of 1922 optimized the distribution of water based on the flows and consumption patterns that prevailed at the time, but it became far from optimal once average precipitation declined, Mexico negotiated a share of the river's flow, and population growth in places like Los Angeles and Las Vegas vastly increased the demand for water.

Ms. Krumme is aware that optimization techniques have brought enormous benefits. Her complaint is that they have also "come with trade-offs: the loss of slack or downtime, the abstracting away of place and particulars." They have "led to a loss of human scale, making events like the Texas-grid breakdown more frequent." One reason why, she laments, is that over time the underlying assumptions behind algorithms fade into obscurity, and certain basic questions are no longer asked.

At times, Ms. Krumme veers toward the romantic, longing for a simpler past. Some readers may find her jabs at "efficiency's overreach" off-putting. Nonetheless, her warning about quantitative excess deserves attention. The basic mathematical and computer skills needed to manipulate large datasets are taught everywhere today, but few of the students who learn these skills receive much grounding in the limitations of the data they are analyzing or the ethical implications of their work. Ms. Krumme's provocative, brisk and refreshingly nontechnical book offers an important corrective.

---

Mr. Levinson's books include "The Box: How the Shipping Container Made the World Smaller and the World Economy Bigger."" [1]

1. Efficient To a Fault. Levinson, Marc.  Wall Street Journal, Eastern edition; New York, N.Y.. 18 Sep 2023: A.17.