AI Has a Lot of Learning to Do to Be Useful

 

"Earlier this year, the PGA Tour's digital chief witnessed ChatGPT make the digital equivalent of a double bogey when the chatbot flubbed a question on basic golf lore: How many times has Tiger Woods won on the Tour?

Generative AI's foundation models can be trained on vast troves of data from the internet and other sources, but still lack deep, specific knowledge even on topics as mainstream as golf, Scott Gutterman, the Tour's senior vice president of digital and broadcast technology, realized.

"There's missing data, there's generalized data. Those things have just kind of led to generalized responses," Gutterman said.

As AI projects creep from the pilot-project stage into operations, corporate users are discovering that many AI models are about as useful out of the box as a new employee entering orientation.

Companies are finding it is critical to augment today's general models, like those offered by Anthropic or OpenAI, with more industry-specific or business-specific data if they are going to be useful. (News Corp, owner of The Wall Street Journal, has a content-licensing partnership with OpenAI.)

But that augmentation presents a spectrum of options, in which higher levels of accuracy and reliability also bring more costs and complexity, said Ritu Jyoti, general manager and group vice president of AI and data as well as global AI lead at research firm International Data Corp. And the augmentation works only if companies have an impeccable handle on their data, Jyoti said.

Yet another question is how much augmentation is sufficient to make models accurate and reliable enough for a specific use. A range of companies including consultants, cloud providers like Amazon Web Services and model makers like OpenAI itself are positioning themselves to help.

IDC estimates that worldwide spending on AI, including AI-enabled applications, infrastructure and related IT and business services, will more than double by 2028 to $632 billion.

The PGA Tour said it is now using an approach known as retrieval augmented generation, or RAG, to avoid any future AI errors. It taps into Claude on the Amazon Web Services infrastructure, then inputs organization-specific information, for instance, a 190-page document containing the Tour's rules. That way, it can ask a query and require the model to directly refer to the information in the document, rather than information culled from the internet.

All outputs are still reviewed by a human before they are delivered to players or customers, Gutterman said.

But the RAG approach can go only so far, according to Will McQueen, vice president and head of data assets and analytics at the agriculture division of pharmaceutical and biotech giant Bayer. "It's, for sure, limited," McQueen said.

It works well enough for certain low-stakes uses, like answering new engineers' questions in the onboarding process, he said. The stakes are much higher, though, when giving farmers advice on tending their crops. For that, the company might go to "fine-tuning," McQueen said, or training parts of the model with proprietary data to get a big leap in accuracy and relevance of responses.

But fine-tuning can be even more expensive, require more-specialized talent -- and still fall short of 100% accuracy, Jyoti said. To help, model makers like OpenAI now offer business customers assistance with model fine-tuning and customization.

The highest level of accuracy available today is only possible when a company trains its own models from the ground up, Jyoti said. But the cost and talent required for that approach are prohibitive to most enterprises, she said.

Custom building just a small language model, for instance, could run anywhere from $500,000 to millions of dollars, said Bayer's McQueen. Continuing maintenance adds another layer of expense.

Even so, Rocket Cos. sees potential in the approach, said Shawn Malhotra, chief technology officer. The lending giant is exploring using an AI model to automatically fill in portions of mortgage applications.

But there is a lot of nuance in the language of homeownership that foundation models on their own wouldn't typically understand, Malhotra said. "It's not simply just the name and number," he said. "You're worried about: What kind of dwelling is this? Is this a detached home?"

Building a small model can help the AI understand those nuances, Malhotra said.

"You may have to give examples of 'when a dwelling is described in the following way, it maps to this kind of property; when it's described in a different way, it's matched to a different kind of property'."

Companies are still learning how to make the various approaches work and figuring out which ones make sense for which situations.

"Most customers are using RAG in one or another fashion," said Sri Elaprolu, director at Amazon.com's AWS GenAI Innovation Center. "Some customers are starting down the fine-tuning route -- we're starting to see that volume increase, rapidly. And more and more customers are exploring what it means to pretrain a model."

Industry-specific models also are emerging to help solve some of the customization complexities.

Legal-tech company Luminance has an AI model that has been trained specifically on over 150 million legal documents over the past 10 years. Getting something wrong in a legal context could be disastrous, said Luminance CEO Eleanor Lightbody. "That's why having AI you've just trained on legal contracts is so, so important."

But the overall supply of these narrow AI models is still limited, said IDC's Jyoti. For now, the task of augmenting and customizing is falling directly to companies.

Like the PGA Tour, which found that ChatGPT would occasionally say Woods has 15 PGA Tour wins. In fact, he has 82.

"Tiger's won 15 majors. That is not the same thing as winning 82 PGA Tour events," said Gutterman. "Beginning to get the models to understand the difference between major wins and PGA Tour wins was something that we saw we would need to do."" [1]

1. AI Has a Lot of Learning to Do to Be Useful. Bousquette, Isabelle.  Wall Street Journal, Eastern edition; New York, N.Y.. 15 Oct 2024: B.1.

Nobelio ekonomikos premija už „įtraukiančią“ laisvąją rinką

 

 "Švedijos karališkoji mokslų akademija Nobelio memorialinę ekonomikos mokslų premiją skyrė trims ekonomistams. Apdovanoti yra Turkijoje gimęs Daronas Acemoglu ir Britanijoje gimęs Simonas Johnsonas, abu iš Masačusetso technologijos instituto, ir Britanijoje gimęs Jamesas A. Robinsonas, Čikagos universiteto ekonomistas ir politologas. Jie gavo apdovanojimą „už tyrimus, kaip formuojasi institucijos ir kaip jos veikia gerovę“.

 

 Ši sritis turi ilgą ir kilnią ekonomikos istoriją. Nobelistų indėlis – išdėstyti empirinius duomenis apie konkrečias ekonomikos institucijas, kurios padėjo arba trukdė ekonomikos augimui, o vėliau – išnagrinėti veiksnius, lėmusius tas institucijas.

 

 Jie pabrėžia, kaip tai padarė Adamas Smithas, kad nuosavybės teisės ir teisinės valstybės principai yra svarbiausi.

 

 Jie teigia, kad vyriausybės gerbia šiuos du ramsčius, nes politinis elitas dalijasi ekonomikos augimo privalumais su „masėmis“, o ne išgauna masių turtus.

 

 Savo 2012 m. knygoje „Why Nations Fail“ ponai Acemoglu ir Robinsonas skirsto šalis į du tipus: išgaunančias ir įtraukias. Išgaunančiose šalyse mažas elitas išgauna turtus iš masių, o įtraukiose šalyse politine valdžia dalijamasi. Kai vyriausybės yra išgaunančios, žmonės turi mažai paskatų gaminti. Tačiau yra priešingai, kai vyriausybės yra įtraukiančios, nes žmonės turi nuosavybės teises ir gali kaupti turtą.

 

 Kodėl politinis elitas kartais pasisako už nuosavybės teises ir teisinę valstybę, o kartais joms prieštarauja? Trijų Nobelistų tyrimai tiria kitų žemynų europinę kolonizaciją. Jie rodo, kad ten, kur santykinai nebuvo ligų, tokių, kaip maliarija, buvo daugiau kolonizatorių. Šių kolonizatorių buvo per daug, kad jie praturtėtų, išnaudodami vietinius gyventojus, todėl jie kūrė, gerovę kuriančias, institucijas. Tačiau ten, kur kolonizatorių mirtingumas buvo didelis, išgyvenę kolonizatoriai tiesiog išgavo turtus iš vietinių gyventojų. Tai paaiškina, kodėl Kanadai ir JAV sekėsi palyginti gerai, kaip kolonijoms, o daugeliui Afrikos ir Lotynų Amerikos šalių sekėsi prastai.

 

 Kaip pažymėjau 2013 m. savo apžvalgoje „Kodėl tautos žlunga“, Adamas Smithas pastebėjo, kad gamtos išteklių būsimoje Kanadoje ir JAV bus mažiau, nei Lotynų Amerikoje. Tačiau ekonominės institucijos, kurias Ispanijos vyriausybė įsteigė Lotynų Amerikoje, buvo mažiau orientuotos į laisvąją rinką ir nuosavybės teises, nei tos, kurias britai įsteigė šiaurinėje Šiaurės Amerikos dalyje. Gaila, kad ponai Acemoglu ir Robinsonas nepaminėjo Smitho įžvalgos. Taip pat jie nepacitavo ekonomisto Mancuro Olsono 1982 metais išleistos knygos „Tautų kilimas ir nuosmukis“, kurioje numatoma nobelistų hipotezė.

 

 Galite pamanyti, kad ponai Acemoglu ir Robinsonas tvirtai tikės ekonomine laisve. Jų darbas atitinka Fraser instituto metinės Pasaulio ekonominės laisvės ataskaitos išvadas, kuriose nustatyta tvirta teigiama koreliacija tarp ekonominės laisvės ir realaus bendrojo vidaus produkto vienam gyventojui. Nors abu autoriai palaiko privačios nuosavybės teises, J. Acemoglu pasisako už didelį minimalų atlyginimą, kuris prisitaiko prie infliacijos. Jis taip pat pritaria griežtiems antimonopoliniams įstatymams.

 

 Už pono Acemoglu tikėjimo antimonopoliniais reikalais slypi jo klaidingas vadinamųjų baronų plėšikų eros aiškinimas. Knygoje „Kodėl tautos žlunga“ ponai Acemoglu ir Robinsonas teigia, kad plėšikai baronai „siekė konsoliduoti monopolijas ir neleisti jokiam potencialiam konkurentui įeiti į rinką ar užsiimti verslu lygiomis sąlygomis“. Ironiška, bet jie išskiria Cornelius Vanderbilt, kaip liūdnai pagarsėjusį baroną plėšiką. Tačiau, būdamas jaunas, Vanderbiltas padėjo jo darbdaviui Thomasui Gibbonsui sulaužyti Aarono Ogdeno tarpvalstybinį monopolį keliauti keltais. Aukščiausiasis Teismas priėmė sprendimą prieš monopoliją byloje Gibbons v. Ogden (1824). Kaip pažymėjo istorikas Burtonas W. Folsomas jaunesnysis savo 1991 metais išleistoje knygoje „Baronų plėšikų mitas“, monopolio žlugimas padidino garlaivių srautą.

 

 Smagu, kad Nobelio premija skiriama ekonomistams, kurie supranta privačios nuosavybės ir teisinės valstybės svarbą. Deja, pono Acemoglu supratimas yra neišsamus. Neseniai jis pasirašė pareiškimą, kuriuo remia Brazilijos vyriausybės žingsnį suvaržyti žodžio laisvę brazilams, norintiems bendrauti, naudojant X. Tik laikas parodys, ar p. Acemoglu pritars tolesniam teisinės valstybės principo mažinimui. Tikėkimės, kad to nepadarys.

 ---

 Ponas Hendersonas yra Stanfordo universiteto Hoover instituto mokslinis bendradarbis ir „Concise Encyclopedia of Economics“ redaktorius.“ [1]

 

Būdamas jaunas, Vanderbiltas dar nebuvo plėšikas. Pasenęs juo tapo.

 

1. A Nobel Prize in Economics for the 'Inclusive' Free Market. Henderson, David R.  Wall Street Journal, Eastern edition; New York, N.Y.. 15 Oct 2024: A.15.

A Nobel Prize in Economics for the 'Inclusive' Free Market

"The Royal Swedish Academy of Sciences awarded the Nobel Memorial Prize in Economic Sciences to three economists. The recipients are Turkish-born Daron Acemoglu and British-born Simon Johnson, both of the Massachusetts Institute of Technology, and British-born James A. Robinson, an economist and political scientist at the University of Chicago. They received the award "for studies of how institutions are formed and affect prosperity."

This field has a long and noble history in economics. The Nobelists' contribution is to lay out empirical data on the specific economic institutions that helped or hindered economic growth and then to examine the factors that led to those institutions. 

They point out, as Adam Smith did, that property rights and the rule of law are key. 

Governments respect these two pillars, they argue, because the political elites share the benefits of economic growth with the "masses" rather than extract the masses' wealth.

In their 2012 book, "Why Nations Fail," Messrs. Acemoglu and Robinson divide countries into two types: extractive and inclusive. In extractive countries, a small elite extracts wealth from the masses, whereas in inclusive countries, political power is shared. When governments are extractive, people have little incentive to produce. But the opposite is true when governments are inclusive, as people have property rights and can accumulate wealth.

Why do political elites sometimes favor property rights and the rule of law and sometimes oppose them? The three Nobelists' research examines European colonization of other continents. They show that where there was a relative absence of diseases, such as malaria, there were more colonizers. These colonizers were too numerous to get rich by exploiting the natives, so they created wealth-building institutions. But where colonizer mortality was high, the colonizers who survived simply extracted wealth from the natives. This explains why Canada and the U.S. did relatively well as colonies and many countries in Africa and Latin America did poorly.

As I noted in my 2013 review of "Why Nations Fail," Adam Smith observed that natural resources were less plentiful in the future Canada and the U.S. than in Latin America. But the economic institutions that Spain's government set up in Latin America were less geared toward the free market and property rights than those that the British set up in the northern part of North America. It's a pity that Messrs. Acemoglu and Robinson didn't cite Smith's insight. Nor did they cite economist Mancur Olson's 1982 book, "The Rise and Decline of Nations," which anticipates the Nobelists' hypothesis.

You might think that Messrs. Acemoglu and Robinson would be strong believers in economic freedom. Their work is consistent with the findings in the Fraser Institute's annual Economic Freedom of the World report, which finds a strong positive correlation between economic freedom and real gross domestic product per capita. While the two authors do favor private property rights, Mr. Acemoglu advocates a high minimum wage that adjusts for inflation. He also favors strong antitrust laws.

Behind Mr. Acemoglu's belief in antitrust is his mistaken interpretation of the era of the so-called robber barons. In "Why Nations Fail," Messrs. Acemoglu and Robinson claim that the robber barons "aimed at consolidating monopolies and preventing any potential competitor from entering the market or doing business on an equal footing." Ironically, they single out Cornelius Vanderbilt as a notorious robber baron. But as a young man, Vanderbilt helped his employer, Thomas Gibbons, break Aaron Ogden's interstate monopoly on ferry travel. The Supreme Court ruled against the monopoly in Gibbons v. Ogden (1824). As historian Burton W. Folsom Jr. noted in his 1991 book, "The Myth of the Robber Barons," the breakdown of the monopoly increased steamboat traffic.

It's good to see a Nobel Prize awarded to economists who understand the importance of private property and the rule of law. Unfortunately, Mr. Acemoglu's understanding is incomplete. He recently signed a statement supporting the Brazilian government's move to rein in freedom of speech for Brazilians who want to communicate using X. Only time will tell whether Mr. Acemoglu will favor further undercutting of the rule of law. Let's hope he doesn't.

---

Mr. Henderson is a research fellow with Stanford University's Hoover Institution and editor of the Concise Encyclopedia of Economics." [1]

As a young man, Vanderbilt was not yet a robber. He became one, getting old.

1. A Nobel Prize in Economics for the 'Inclusive' Free Market. Henderson, David R.  Wall Street Journal, Eastern edition; New York, N.Y.. 15 Oct 2024: A.15.