Turing Award Goes to 2 Pioneers of Artificial Intelligence

 

"Andrew Barto and Richard Sutton developed reinforcement learning, a technique vital to chatbots like ChatGPT.

In 1977, Andrew Barto, as a researcher at the University of Massachusetts, Amherst, began exploring a new theory that neurons behaved like hedonists. The basic idea was that the human brain was driven by billions of nerve cells that were each trying to maximize pleasure and minimize pain.

A year later, he was joined by another young researcher, Richard Sutton. Together, they worked to explain human intelligence using this simple concept and applied it to artificial intelligence.

The result was “reinforcement learning,” a way for A.I. systems to learn from the digital equivalent of pleasure and pain.

On Wednesday, the Association for Computing Machinery, the world’s largest society of computing professionals, announced that Dr. Barto and Dr. Sutton had won this year’s Turing Award for their work on reinforcement learning. The Turing Award, which was introduced in 1966, is often called the Nobel Prize of computing. The two scientists will share the $1 million prize that comes with the award.

Over the past decade, reinforcement learning has played a vital role in the rise of artificial intelligence, including breakthrough technologies such as Google’s AlphaGo and OpenAI’s ChatGPT. The techniques that powered these systems were rooted in the work of Dr. Barto and Dr. Sutton.

“They are the undisputed pioneers of reinforcement learning,” said Oren Etzioni, a professor emeritus of computer science at the University of Washington and founding chief executive of the Allen Institute for Artificial Intelligence. “They generated the key ideas — and they wrote the book on the subject.”

Their book, “Reinforcement Learning: An Introduction,” which was published in 1998, remains the definitive exploration of an idea that many experts say is only beginning to realize its potential.

Psychologists have long studied the ways that humans and animals learn from their experiences. In the 1940s, the pioneering British computer scientist Alan Turing suggested that machines could learn in much the same way.

But it was Dr. Barto and Dr. Sutton who began exploring the mathematics of how this might work, building on a theory that A. Harry Klopf, a computer scientist working for the government, had proposed. Dr. Barto went on to build a lab at UMass Amherst dedicated to the idea, while Dr. Sutton founded a similar kind of lab at the University of Alberta in Canada.

“It is kind of an obvious idea when you’re talking about humans and animals,” said Dr. Sutton, who is also a research scientist at Keen Technologies, an A.I. start-up, and a fellow at the Alberta Machine Intelligence Institute, one of Canada’s three national A.I. labs. “As we revived it, it was about machines.”

This remained an academic pursuit until the arrival of AlphaGo in 2016. Most experts believed that another 10 years would pass before anyone built an A.I. system that could beat the world’s best players at the game of Go.

But during a match in Seoul, South Korea, AlphaGo beat Lee Sedol, the best Go player of the past decade.

The trick was that the system had played millions of games against itself, learning by trial and error. It learned which moves brought success (pleasure) and which brought failure (pain).

The Google team that built the system was led by David Silver, a researcher who had studied reinforcement learning under Dr. Sutton at the University of Alberta.

Many experts still question whether reinforcement learning could work outside of games. Game winnings are determined by points, which makes it easy for machines to distinguish between success and failure.

But reinforcement learning has also played an essential role in online chatbots.

Leading up to the release of ChatGPT in the fall of 2022, OpenAI hired hundreds of people to use an early version and provide precise suggestions that could hone its skills. They showed the chatbot how to respond to particular questions, rated its responses and corrected its mistakes. By analyzing those suggestions, ChatGPT learned to be a better chatbot.

Researchers call this “reinforcement learning from human feedback,” or R.L.H.F. And it is one of the key reasons that today’s chatbots respond in surprisingly lifelike ways.

(The New York Times has sued OpenAI and its partner, Microsoft, for copyright infringement of news content related to A.I. systems. OpenAI and Microsoft have denied those claims.)

More recently, companies like OpenAI and the Chinese start-up DeepSeek have developed a form of reinforcement learning that allows chatbots to learn from themselves — much as AlphaGo did. By working through various math problems, for instance, a chatbot can learn which methods lead to the right answer and which do not.

If it repeats this process with an enormously large set of problems, the bot can learn to mimic the way humans reason — at least in some ways. The result is so-called reasoning systems like OpenAI’s o1 or DeepSeek’s R1.

Dr. Barto and Dr. Sutton say these systems hint at the ways machines will learn in the future. Eventually, they say, robots imbued with A.I. will learn from trial and error in the real world, as humans and animals do.

“Learning to control a body through reinforcement learning — that is a very natural thing,” Dr. Barto said.” [1]

1. Turing Award Goes to 2 Pioneers of Artificial Intelligence. Metz, Cade.  New York Times (Online) New York Times Company. Mar 5, 2025.
  

 

Kilimas aukštyn: kaip Trumpas planuoja supurtyti aukštąjį mokslą

 

 „Praėjusį mėnesį D. Trumpo administracijos paskelbtas įrašas socialinėje žiniasklaidoje sukėlė alpimo priepuolius visoje akademijoje. Nacionaliniai sveikatos institutai (NIH), finansuojantys biomedicinos tyrimus, paskelbė, kad mažina pinigų sumą, kurią vyriausybė moka dotacijų gavėjams už pridėtines išlaidas.

 

 Pasak NIH, 9 milijardai JAV dolerių iš 35 milijardų JAV dolerių, kuriuos pernai skyrė tyrimams, "buvo panaudoti pridėtinėms administracinėms išlaidoms, vadinamoms "netiesioginėmis išlaidomis". "Mokykla, kuri gauna dotaciją, paprastai gauna papildomus 50% pakeistų visų tiesioginių išlaidų (įskaitant atlyginimus administratoriams, medžiagas ir reikmenis, paslaugas, keliones ir kai kuriuos mokėjimus už subrangos sutartis), kad padengtų šias administracines išlaidas. Prestižinėse mokyklose, tokiose, kaip Harvardas, pridėtinės išlaidos – už naudojimąsi pastatais, elektrą, pagalbinį personalą ir kt. – gali siekti net 70%. Trumpo administracija nori apriboti šį skaičių iki 15%, kas, jos vertinimu, mokesčių mokėtojams sutaupys daugiau, nei 4 mlrd. dolerių.

 

 Darbo ekonomistas Richardas Vedderis mano, kad tai yra sistemos šokas, kurio reikia aukštajam mokslui. „Žinoma, universitetai, turintys dideles mokslinių tyrimų dotacijas, dėl to eina iš proto“, – sakė jis. "Tačiau jei kalbate su kuo nors universitete, žinote, kad šios pridėtinės išlaidos yra labai išpūstos, palyginti su tikrosiomis ribinėmis arba papildomomis išlaidomis universiteto, atliekančio tyrimus." Jis pridūrė, kad daugelis mokyklų surenka tiek daug pridėtinių pinigų, kad dalį jų grąžina mokslininkams kaip paskatą prašyti daugiau mokslinių tyrimų dotacijų. „Tai savotiškas sukčių žaidimas, pagrįstas klaidingomis prielaidomis ir klaidinga ekonomika“, – sako ponas Vedderis. Neabejotinas vienodas tarifas būtų daug veiksmingesnis.

 

 Būsimoje knygoje „Let Colleges Fail: The Power of Creative Destruction in Higher Education“ p. Vedderis teigia, kad viena didžiausių šiandieninių aukštųjų mokyklų problemų yra ta, kad kolegijos nėra pakankamai drausminamos rinkos jėgų. Dėl to vyriausybė subsidijuoja per daug administracinio išsipūtimo. Jo paantraštė yra nuoroda į laisvosios rinkos ekonomistą Josephą Schumpeterį (1883–1950), kuris kapitalizmą apibūdino, kaip „kūrybinio naikinimo“ procesą, kurio metu rinkos perskirsto išteklius iš neproduktyvaus į produktyvų naudojimą. „Tai gana gerai pasiteisino Amerikos verslui“, – sakė J. Vedderis. "Kodėl mes neturime jo aukštesniajam išsilavinimui?"

 

 Knygoje aiškinama, kad viena problema yra ta, kad universitetai iš esmės yra valstybės išlaikomos. „Kolegijose ir universitetuose dominuoja žmonės, veikiantys už įprastos, į pelną orientuotos, privačios rinkos ekonomikos ribų“, – rašo J. Vedderis. Jo skaičiavimais, universiteto darbuotojų produktyvumas per pastaruosius 50 metų sumažėjo ne tik lyginant su vidutiniu JAV darbuotojo, bet ir absoliučiu dydžiu. 2021 m. kolegijos studentui parengti prireikė daugiau dėstytojų ir darbuotojų, nei 1972 m.

 

 Realybė yra tokia, kad beveik visur, kur žiūrite į aukštąjį išsilavinimą, yra neefektyvumo. Pradėkite nuo mūsų pernelyg dosnių studentų paskolų programų, dėl kurių mokslo kaina išaugo greičiau, nei infliacija, nes kolegijos žino, kad vyriausybė padengia didžiąją dalį išlaidų. Daug daugiau žmonių šiandien lanko koledžą, nei gali pasinaudoti šia patirtimi, sprendžiant iš maždaug 40 % nebaigusių studijų ir į tai, kad tiek daug diplomą įgijusių žmonių baigia su darbu, kuriam jo nereikia.

 

 Tuo tarpu neturtingųjų – numatomų studentų paskolų gavėjų – galimybė studijuoti koledže per dešimtmečius beveik nepakito. P. Vedderis praneša, kad absolventų, kurie patenka iš apatinių 25% pajamų pasiskirstymo, procentas yra panašus į 1970 m. Jei kolegijos nori priimti studentus, nepaisant jų sėkmės galimybių, mokykloms suteikite „odą žaidime“, rašo jis. Priverskite kolegijas „dalytis paskolos negrąžinimo išlaidomis, jei jos priima daug vidutinių studentų, kurie meta studijas arba negrąžina paskolos studentams“.

 

 Nereikia būti aukštojo mokslo ekonomikos ekspertu, kad suprastum, jog akademinė bendruomenė nyksta. 2023 m. Nacionalinio nuomonės tyrimų centro ir The Wall Street Journal atliktoje apklausoje 56 % respondentų teigė, kad koledžas „nebuvo vertas išlaidų, nes žmonės dažnai baigia studijas, neturėdami specifinių darbo įgūdžių ir turėdami daug skolų, kurias reikia sumokėti“. Dar blogiau, 18–34 metų amžiaus žmonės, kurie neseniai studijavo koledže, rečiau, nei vyresnio amžiaus žmonės sako, kad studijų kaina buvo to verta.

 

 „Jei aukštasis mokslas galėtų kažkaip pakeisti savo būdus ir geriau pamėgdžioti į rinką orientuotą privačią ekonomiką“, – rašo ponas Vedderis, „tai pagerintų kai kurias su ja susiduriančias ligas“ – didėjančias išlaidas, mažėjantį studentų skaičių, mažą visuomenės paramą – „ir galbūt net išspręstų tai, ką dabar laikau didžiausia problema: šokiruojantį intelektualinės raiškos aplinkos nuosmukį ir sunkesnę raiškos įvairovę.” [1]

1.  Upward Mobility: How Trump Plans to Shake Up Higher Education. Riley, Jason L.  Wall Street Journal, Eastern edition; New York, N.Y.. 05 Mar 2025: A15.

 

Upward Mobility: How Trump Plans to Shake Up Higher Education

"A social-media post last month from the Trump administration triggered fainting spells throughout the academy. The National Institutes of Health, which funds biomedical research, announced that it is reducing the amount of money the government pays grant recipients for overhead costs.

According to NIH, $9 billion of the $35 billion that it granted for research last year "was used for administrative overhead, what is known as 'indirect costs.' " A school that receives a grant typically gets an additional 50% of its modified total direct costs (which includes salaries, materials and supplies, services, travel and some subcontract payments) to cover these administrative expenses. At prestige schools such as Harvard, the overhead payments -- for the use of buildings, electricity, support staff, etc. -- can run as high as 70%. The Trump administration wants to cap this figure at 15%, which it estimates will save taxpayers more than $4 billion annually.

The labor economist Richard Vedder thinks this is exactly the shock to the system that higher education needs. "Of course the universities with heavy research grants are going crazy over this," he told me. "But if you talk to anyone at a university, you know that those overhead costs are vastly inflated compared with the true marginal cost, or extra cost, to the university doing the research." He added that many schools collect so much overhead money that they give some of it back to researchers as an incentive to apply for more research grants. "It's kind of a con game, all based on false assumptions and faulty economics," Mr. Vedder says. A nonnegotiable uniform rate would be far more efficient.

In a forthcoming book, "Let Colleges Fail: The Power of Creative Destruction in Higher Education," Mr. Vedder argues that one of the biggest problems with higher ed today is that colleges aren't sufficiently disciplined by market forces. The result is too much administrative bloat subsidized by the government. His subtitle is a reference to the free-market economist Joseph Schumpeter (1883-1950), who described capitalism as a process of "creative destruction" whereby markets reallocate resources from unproductive to productive uses. "It's worked pretty well for American business," Mr. Vedder said. "Why don't we have it for higher ed?"

One problem, the book explains, is that universities are essentially wards of the state. "Colleges and universities are dominated by people operating outside of the normal profit-oriented private market economy," Mr. Vedder writes. By his calculations, the productivity of university employees over the past 50 years has declined not only in comparison with the average U.S. worker but also in absolute terms. It took more faculty and staff to educate a college student in 2021 than it did in 1972.

The reality is that there are inefficiencies almost everywhere you look in higher ed. Start with our overly generous student-loan programs, which have caused tuition to rise faster than inflation because colleges know that the government is absorbing most of the cost. Far more people attend college today than stand to benefit from the experience, judging from the roughly 40% who don't graduate and the fact that so many people who do get a degree wind up in jobs that don't require one.

Meanwhile, college access for the poor -- the intended beneficiaries of student loans -- has barely budged over the decades. Mr. Vedder reports that the percentage of graduates who come from the bottom 25% of the income distribution is similar to what it was in 1970. If colleges want to accept students regardless of their chances of success, give schools "skin in the game," he writes. Force colleges "to share in the costs of loan forfeiture if they accept a lot of mediocre students who drop out or fail to repay their student loans."

You don't need to be an expert in the economics of higher education to understand that academia is in decline. In a 2023 poll conducted by the National Opinion Research Center and The Wall Street Journal, 56% of respondents said that college was "not worth the cost because people often graduate without specific job skills and with a large amount of debt to pay off." Worse, people who have most recently experienced college, those 18 to 34, are less likely than older cohorts to say the cost of college was worth it.

"If higher education can somehow change its ways and better emulate the market-oriented private economy," Mr. Vedder writes, "it would improve some of the maladies facing it" -- rising costs, falling enrollment, low public support -- "and possibly even deal with what I now view as the greatest problem: a shocking decline in intellectual diversity and a worsening environment conducive to free expression and civil debate."" [1]

1.  Upward Mobility: How Trump Plans to Shake Up Higher Education. Riley, Jason L.  Wall Street Journal, Eastern edition; New York, N.Y.. 05 Mar 2025: A15.

 

Putin Played the Long Game. It Seems to Be Paying Off

 

"More than a decade before Russia's armed forces poured over the border into Ukraine, President Vladimir Putin stood before world leaders and delivered a long, icy speech demanding a radical overhaul of the world order.

"We have reached that decisive moment when we must seriously think about the architecture of global security," Putin said in 2007 speech in Munich, accusing the North Atlantic Treaty Organization of breaking a promise by expanding into Eastern Europe, and calling for an end to U.S. hegemony.

Tensions between Moscow and the West grew in the years that followed. Russia sent its military into Georgia, Syria and Ukraine. The 2022 invasion of Ukraine spurred a broad Western effort to isolate Moscow and pushed new countries into NATO.

Putin dug in as his military suffered battlefield setbacks and his economy was squeezed by Western sanctions. He played the long game. Now, that perseverance appears to be paying off as the world shifts decisively in his direction. The U.S. has paused military aid to Ukraine and called for an end to Moscow's isolation. It is distancing itself from traditional allies in Europe.

"We all see how rapidly the world is changing," Putin told his security services on Thursday, after a U.S.-Russia meeting in Saudi Arabia. Moscow and Washington, he said, are now ready to address "strategic problems in the architecture of the world."

Even Putin's most hawkish advisers have been surprised by the speed with which the tone coming from the White House has changed, say people who travel to Moscow and speak with Russian officials.

"The new administration is rapidly changing all foreign policy configurations," Putin's spokesman Dmitry Peskov said of President Trump's team.

Trump, who had been calling for both sides to end the war, recently turned his attention decisively to Ukraine. He called Ukrainian President Volodymyr Zelensky a dictator and blamed him for starting the war, echoing Moscow. That culminated in an on-camera clash Friday between the Ukrainian leader and Trump.

In Munich last month, Vice President JD Vance said the erosion of democracy in Europe posed a greater threat to the continent than Russia or China -- a common Putin claim.

"We haven't seen this before," said Sergey Radchenko, a Russia historian and author of a new book on Moscow's Cold War strategy. "Not just the political realignment, but the alignment of values."

For Putin, the moment is also a vindication of the patient strategy he has honed during a quarter-century in power. The former KGB agent elevated from obscurity to lead Russia at the turn of the millennium has railed passionately against the U.S.-led world order ushered in by the Soviet collapse in 1991, which Putin has called "the greatest geopolitical catastrophe of the century."

The sentiments Putin voiced in Munich stem from grievances toward the U.S. that deepened in 2004, according to Thomas Graham, a former White House adviser on Russia to George W. Bush. That year, a Western-backed revolution convulsed Ukraine, and Chechen separatists stormed a school in Russia's North Caucasus region. Putin blamed the U.S. for encouraging the separatist movement.

"Those two events led Putin to believe that the United States really wasn't interested in a partnership with Russia, that counterterrorism and democracy promotion were really just smokescreens for America's geopolitical advance into the former Soviet space at Russia's expense," said Graham. "He concluded at that point that the goal of the United States was really to erode Russia's standing as a great power."

Putin's 2007 speech made clear for the first time the depth of his anger about perceived U.S. arrogance. But many Western officials appeared to dismiss Putin's warning at the time.

"One Cold War was quite enough," Robert Gates, the U.S. defense secretary at the time, said in response.

The following year Russia was in conflict with Georgia and seized control of two pro-Russian enclaves in that ex-Soviet republic, prompting no meaningful Western response. The administration of Barack Obama sought a "reset" with Moscow under Russia's interim President Dmitry Medvedev, but Putin's return to the presidency in 2012 was followed by a crackdown on dissent and deepening suspicion of the West.

Relations tanked when Putin reunited with Crimea in 2014 and sent his army into eastern Ukraine.

In response to events in Ukraine in 2022, former President Joe Biden hit Moscow with sanctions and promised to support Ukraine "for as long as it takes."

After Trump's victory last year, Putin launched a charm offensive, echoing statements about the 2020 election and praising Trump's response to the attempt on his life in July.

Now, with Trump bringing Moscow in from the cold and suspending crucial military assistance to Kyiv, he sees an opportunity to fundamentally change Russia's position in the world, analysts say.

What he wants is far more than a simple settlement to end the fighting. Putin's goal is to turn Ukraine into a neutered state, and bar the country from rearming with Western support. His ambition is to force NATO out of Eastern Europe.

Radchenko said the current moment has historical parallels to the period following World War II, when Soviet dictator Joseph Stalin sought U.S. agreement to divide Europe into spheres of influence. The U.S. instead remained engaged in the continent, acting as a guarantor of security, and limiting Stalin's ambitions. Today, he says, the Kremlin is advancing a similar vision. "But what I find remarkable is that it's the Trump administration that is now also embracing this vision of the world."" [1]

 
1. World News: Putin Played the Long Game. It Seems to Be Paying Off. Luxmoore, Matthew.  Wall Street Journal, Eastern edition; New York, N.Y.. 05 Mar 2025: A6.