Artificial Intelligence (AI): The Cloud May Not Be Ready for the Boom in AI.

"Many companies say the cloud is their go-to when it comes to training and running large AI applications -- but today, only a small portion of existing cloud infrastructure is actually set up to support that. The rest is not.

Now cloud providers, including Amazon Web Services, Microsoft Azure and Google Cloud are under pressure to change that calculus to meet the computing demands of a major AI boom -- and as other hardware providers see a potential opening.

"There's a pretty big imbalance between demand and supply at the moment," said Chetan Kapoor, director of product management at Amazon Web Services' Elastic Compute Cloud division.

Most generative AI models today are trained and run in the cloud. These models, designed to generate original text and analysis, can be anywhere from 10 times to a 100 times bigger than older AI models, said Ziad Asghar, senior vice president of product management at Qualcomm Technologies, adding that the number of use cases as well as the number of users are also exploding.

"There is insatiable demand," for running large language models right now, including in industry sectors like manufacturing and finance, said Nidhi Chappell, general manager of Azure AI Infrastructure.

It is putting more pressure than ever on a limited amount of computing capacity that relies on an even more limited number of specialized chips, such as graphic chips, or GPUs, from Nvidia. Companies like Johnson & Johnson, Visa, Chevron and others all said they anticipate using cloud providers for generative AI-related use cases.

But much of the infrastructure wasn't built for running such large and complex systems. Cloud sold itself as a convenient replacement for on-premise servers that could easily scale up and down capacity with a pay-as-you-go pricing model. Much of today's cloud footprint consists of servers designed to run multiple workloads at the same time that leverage general-purpose CPU chips.

A minority of it, according to analysts, runs on chips optimized for AI, such as GPUs and servers designed to function in collaborative clusters to support bigger workloads, including large AI models. GPUs are better for AI since they can handle many computations at once, whereas CPUs handle fewer computations simultaneously.

At AWS, one cluster can contain up to 20,000 GPUs. AI-optimized infrastructure is a small percentage of the company's overall cloud footprint, said Kapoor, but it is growing at a much faster rate. He said the company plans to deploy multiple AI-optimized server clusters over the next 12 months.

Microsoft Azure and Google Cloud Platform said they are similarly working to make AI infrastructure a greater part of their overall fleets. However, Microsoft's Chappell said that that doesn't mean the company is necessarily moving away from the shared server -- general purpose computing -- which is still valuable for companies.

Other hardware providers have an opportunity to make a play here, said Lee Sustar, principal analyst at tech research and advisory firm Forrester, covering public cloud computing for the enterprise.

Dell Technologies expects that high cloud costs, linked to heavy use -- including training models -- could push some companies to consider on-premises deployments. The computer maker has a server designed for that use.

"The existing economic models of primarily the public cloud environment weren't really optimized for the kind of demand and activity level that we're going to see as people move into these AI systems," Dell's Global Chief Technology Officer John Roese said.

On premises, companies could save on costs like networking and data storage, Roese said.

Cloud providers said they have several offerings available at different costs and that in the long term, on-premises deployments could end up costing more because enterprises would have to make huge investments when they want to upgrade hardware.

Qualcomm said that in some cases it might be cheaper and faster for companies to run models on individual devices, taking some pressure off the cloud. The company is currently working to equip devices with the ability to run larger and larger models.

And Hewlett Packard Enterprise is rolling out its own public cloud service, powered by a supercomputer, that will be available to enterprises looking to train generative AI models in the second half of 2023. Like some of the newer cloud infrastructure, it has the advantage of being purposely built for large-scale AI use cases, said Justin Hotard, executive vice president and general manager of High Performance Computing, AI & Labs." [1]

1. Artificial Intelligence: The Cloud May Not Be Ready for the Boom in AI. Bousquette, Isabelle. 
Wall Street Journal, Eastern edition; New York, N.Y. [New York, N.Y]. 11 July 2023: B.4.

 

Dirbtinis intelektas (AI): debesis gali būti nepasirengęs dirbtinio intelekto klestėjimui

 

     "Daugelis kompanijų teigia, kad debesys yra jų pagrindinis tikslas, kai kalbama apie mokymą ir didelių AI programų paleidimą, tačiau šiandien tik nedidelė esamos debesų infrastruktūros dalis yra sukurta tam, kad AI paremtų. Likusioji - ne.

 

     Dabar debesų paslaugų teikėjai, įskaitant „Amazon Web Services“, „Microsoft Azure“ ir „Google Cloud“, patiria spaudimą pakeisti šį skaičiavimą, kad atitiktų didelio AI bumo skaičiavimo poreikius, o kiti aparatinės įrangos tiekėjai mato potencialias galimybes.

 

     „Šiuo metu yra gana didelis disbalansas tarp paklausos ir pasiūlos“, – sakė Chetanas Kapooras, „Amazon Web Services“ Elastic Compute Cloud skyriaus produktų valdymo direktorius.

 

     Dauguma generuojančiųjų AI modelių šiandien yra mokomi ir veikia debesyje. Šie modeliai, sukurti originaliam tekstui ir analizei generuoti, gali būti nuo 10 iki 100 kartų didesni už senesnius AI modelius, sakė Ziadas Asgharas, „Qualcomm Technologies“ produktų valdymo vyresnysis viceprezidentas ir pridūrė, kad naudojimo atvejų skaičius taip pat kyla, nes vartotojų skaičius taip pat sparčiai auga.

 

     „Šiuo metu yra nepasotinama paklausa“ naudoti didelius kalbos modelius, įskaitant pramonės sektorius, tokius, kaip gamyba ir finansai, sakė Nidhi Chappell, Azure AI infrastruktūros generalinis direktorius.

 

     Tai daro didesnį spaudimą, nei bet kada anksčiau, ribotam skaičiavimo pajėgumui, kuris priklauso nuo dar riboto skaičiaus specializuotų lustų, tokių, kaip „Nvidia“ grafiniai lustai arba GPU. Tokios įmonės, kaip „Johnson & Johnson“, „Visa“, „Chevron“ ir kitos teigė, kad tikisi naudoti debesų paslaugų teikėjus generatyviems su AI susijusiems naudojimo atvejams.

 

     Tačiau didelė dalis infrastruktūros nebuvo sukurta tokioms didelėms ir sudėtingoms sistemoms valdyti. „Debesis“ buvo parduotas, kaip patogus vietinių serverių pakaitalas, kuris galėtų lengvai padidinti ir sumažinti pajėgumą, taikant „mokėk tiek, kiek naudoji“ kainodaros modelį. Didžiąją šiandienos debesų srauto dalį sudaro serveriai, skirti vykdyti kelis darbo krūvius tuo pačiu metu ir kurie naudoja bendrosios paskirties procesoriaus lustus.

 

     Anot analitikų, mažuma jo veikia, naudojant dirbtiniam intelektui optimizuotus lustus, pvz., GPU ir serverius, sukurtus veikti bendradarbiavimo grupėse ir palaikyti didesnį darbo krūvį, įskaitant didelius AI modelius. GPU yra geresni dirbtiniam intelektui, nes jie vienu metu gali atlikti daug skaičiavimų, o CPU vienu metu atlieka mažiau skaičiavimų.

 

     AWS viename klasteryje gali būti iki 20 000 GPU. Dirbtinio intelekto optimizuota infrastruktūra sudaro nedidelę bendros bendrovės debesų pėdsakų dalį, sakė Kapooras, tačiau ji auga daug greičiau. Jis sakė, kad bendrovė per ateinančius 12 mėnesių planuoja įdiegti keletą AI optimizuotų serverių grupių.

 

     „Microsoft Azure“ ir „Google Cloud Platform“ teigė, kad jie taip pat stengiasi, kad dirbtinio intelekto infrastruktūra taptų didesne jų bendro priemonių parko dalimi. Tačiau „Microsoft“ Chappell teigė, kad tai nereiškia, kad įmonė būtinai tolsta nuo bendro serverio – bendrosios paskirties skaičiavimo, kuris vis dar yra vertingas įmonėms.

 

     Kiti techninės įrangos tiekėjai turi galimybę žaisti čia, sakė Lee Sustar, technologijų tyrimų ir konsultavimo įmonės „Forrester“, apimančios viešąją debesų kompiuteriją, vyriausiasis analitikas.

 

     „Dell Technologies“ tikisi, kad didelės debesų sąnaudos, susijusios su intensyviu naudojimu, įskaitant mokymo modelius, gali paskatinti kai kurias įmones apsvarstyti galimybę diegti vietoje. Kompiuterių gamintojas turi tam skirtą serverį.

 

     „Esami ekonominiai modeliai, daugiausia viešosios debesų aplinkos, nebuvo iš tikrųjų optimizuoti tokiam poreikiui ir veiklos lygiui, kokį matysime, žmonėms pereinant prie šių dirbtinio intelekto sistemų“, – sakė „Dell“ pasaulinis technologijų vadovas Johnas Roese'as.

 

     Patalpose įmonės galėtų sutaupyti tokių išlaidų, kaip tinklų kūrimas ir duomenų saugojimas, sakė Roese.

 

     Debesų paslaugų teikėjai teigė, kad jie turi keletą pasiūlymų už skirtingą kainą ir kad ilgainiui vietinis diegimas gali kainuoti daugiau, nes įmonės turės daug investuoti, norėdamos atnaujinti aparatinę įrangą.

 

     „Qualcomm“ teigė, kad kai kuriais atvejais įmonėms gali būti pigiau ir greičiau paleisti modelius atskiruose įrenginiuose, sumažinant debesies spaudimą. Šiuo metu įmonė stengiasi aprūpinti įrenginius, galinčius paleisti vis didesnius modelius.

 

     O „Hewlett Packard Enterprise“ diegia savo viešąją debesijos paslaugą, kurią maitina superkompiuteris, kuri bus prieinama įmonėms, norinčioms mokyti generuojančius dirbtinio intelekto modelius antroje 2023 m. pusėje. Kaip ir kai kurioms naujesnėms debesų infrastruktūroms, ji turi pranašumą: yra specialiai sukurta didelio masto AI naudojimo atvejams, sakė Justinas Hotardas, vykdomasis viceprezidentas ir High Performance Computing, AI & Labs generalinis direktorius." [1]

1. Artificial Intelligence: The Cloud May Not Be Ready for the Boom in AI. Bousquette, Isabelle. 
Wall Street Journal, Eastern edition; New York, N.Y. [New York, N.Y]. 11 July 2023: B.4.

 

Lego-Style 'Chiplets' Stack Together To Drive Faster Artificial Intelligence.

"The artificial-intelligence boom is pushing chip makers to speed up development of designs that stack chips together like high-tech Lego pieces.

"Chiplets" can be an easier way to design more powerful chips, according to industry executives who call the technology one of the most significant advances since the dawn of the integrated circuit more than 60 years ago.

"A huge part of the future of semiconductors is packaging and chiplets," said International Business Machines head of research Dario Gil in an interview. "It's just much more powerful than having to design a massive chip from scratch."

Tech giants including Advanced Micro Devices, Intel, Microsoft, Qualcomm, Samsung Electronics and Taiwan Semiconductor Manufacturing Co. last year formed a coalition to create standards for designing chiplets. Nvidia, which has ridden the AI wave to become the world's first trillion-dollar chip company, joined later. IBM and some Chinese companies are members too.

Apple's high-end Mac Studio computer, introduced last year and updated in June, is among the early consumer products using a form of chiplet technology to connect two computing processors. The chips are made by TSMC. In recent months, Intel and Nvidia have each announced chiplet-based customized products.

A typical consumer device such as a smartphone contains many types of chip for functions including data processing, graphics processing, memory, telecommunications and power control. The chips are delicately tethered to minuscule wires and ensconced in a protective plastic casing, forming a package that can be fixed to a circuit board.

With the new chiplet packaging, engineers have found ways to bolt together pre-existing chips, the equivalent of using a few Lego pieces to build a toy car.

Wang Xiaoyang, an executive at Chinese chiplet startup M Square, compared chip designers to creators of recipes and chiplets to pre-prepared ingredients. Chip-design companies can toss together the ingredients they want, "and it's simple to cook up the dish and serve it right away on the table," he said in a company release.

The concept is particularly appealing to AI companies that are in a hurry to design chips optimized for the type of calculations used in AI.

Nvidia said its chiplet technology allowed its existing products such as graphics-processing chips to be connected with custom chips designed by companies with specialized needs.

As packing more transistors into a tiny space becomes ever more difficult, the stacking of chips "will be the primary mechanism for continuing to scale chip performance in a cost- and power-efficient way," Nvidia said in a report last year to the Commerce Department.

TSMC, the world's largest contract maker of chips, has its own platforms for clients to design chiplet-based products. The company, whose clients include Apple, said it expects its floor space for production in advanced packaging in 2025 to be twice what it was in 2021.

Companies are still working to bring down chiplets' production cost, with work continuing on how to stitch them together most effectively. Also, chiplets need a different process to verify performance, and they aren't suited to every function. People in the industry say chiplet designs are a good fit for products such as higher-end Apple desktops that sell for $4,000 and up, as opposed to being the main chips in current mass-market smartphones." [1]

1. Lego-Style 'Chiplets' Stack Together To Drive Faster Artificial Intelligence. Yang, Jie. 
Wall Street Journal, Eastern edition; New York, N.Y. [New York, N.Y]. 11 July 2023: B.1.