The Age of the Megachip Is Dawning --- Mini-metropolises of silicon are coming to our devices

If microchips were cities, the new, industrywide strategy for making them better could be summed up in one word: sprawl. In some case, the chips inside our most powerful devices are taking up so much real estate they hardly qualify as "micro" anymore.

One way engineers are making this happen is by piling microchips atop one another. It's like urban infill, only instead of building towering new apartment blocks, the usually pancake-flat tiles of silicon inside of computers are becoming multistory, with the circuitry used for functions such as memory, power management and graphics stacked on top of each other.

What's driving this trend in chip design is a simple reality: The pressure to continue making chips faster and our devices more capable is unrelenting, and the chip industry's ability to keep pace by shrinking transistors to eke out more performance is running into technical barriers.

As a result, semiconductor engineers are increasing performance by jamming chips closer together. What's emerging are mini-metropolises of silicon at the heart of our electronic world. In some cases they are growing so big they are attaining physical dimensions rarely seen before in chips.

Currently, most chips are around the size of a dime or a quarter, but some chips are now growing to nearly the size of a playing card, or in one case, a dinner plate.

These megachips are showing up not just in the world's most powerful supercomputers, but also in devices found at home. Microsoft's Xbox videogame console and Sony's PlayStation 5 use some, designed by Advanced Micro Devices. And Apple embraced this approach in its M1 Ultra for its Mac Studio computers.

But these megachips can present engineers with challenges when it comes to managing the extra heat they create from all the calculations being performed in densely packed circuits. And though they can be more energy efficient, their sheer size means they sometimes end up using a lot of power. If you are wondering why megachips aren't yet in your mobile device, this is your answer.

By some measures, megachips are simply a way to continue the trend of Moore's Law -- the observation by Intel founder Gordon Moore that every two years or so consumers can expect about twice as many transistors, and therefore computing power, per dollar. This rule of thumb has been declared at an end before, but chips just keep getting better. Megachips are just the industry's latest innovation to deliver on the promise of ever-higher performance.

Making megachips is no small feat, in part because doing so means maneuvering each chip component into place with nanoscale precision, and connecting them without the benefit of a microscopic soldering gun.

This is now possible in large part because of recent innovation in an area that has long been neglected by the chip industry. That area is "packaging." That's the usually obscure step that comes after a microchip has been manufactured, when it is connected to tiny wires and enveloped in plastic before being placed on a board, also covered in wires, that connects it to the rest of a device.

In traditional devices, a chip that receives and transmits radio waves (say, to communicate via Wi-Fi) may connect to another one doing general-purpose computations, and the connection between them is something literally called a "bus." But like its real-world equivalent, this bus is hardly a fast way to transport anything between these adjacent silicon cities. The new packaging of megachips instead connects these two chips -- and potentially many more -- directly. The result is more like putting all of these chips together under one roof, in a single high-rise.

A conventional microchip must devote nearly a third of its area -- and as much of its power consumption -- to the circuits that communicate the results of the chip's calculations to the rest of a device, says Subramanian Iyer, former director of packaging development at International Business Machines, and now a professor at the University of California, Los Angeles. Stacking chips makes communication between them faster because it allows many more connections between them.

The essential building block to make chip stacking happen is a new kind of microchip, called a "chiplet." It does away with some of the old-style circuitry to communicate more directly with other chiplets. By creating many short, direct connections -- often forged from the same silicon that the chips themselves are made from, rather than copper or some other metal -- these chiplets can be fused with other chiplets to form megachips.

Direct communication between the different chiplets that together make up a megachip is what allows them to function like a single, giant microprocessor, says Rakesh Kumar, a professor of electrical engineering at the University of Illinois Urbana-Champaign.

One extreme example is Intel's recently announced Ponte Vecchio graphics processors. Each is made up of 63 different chiplets. These chiplets, stacked on top of and crammed next to one another, have a total area of 3,100 square millimeters, and include 100 billion transistors. For comparison, the typical chip at the heart of a laptop measures less than 150 square millimeters, or about 1/20th the size, and has about 1.5 billion transistors -- 1.5% as many.

AMD, which pioneered the current era of chiplet technology, already offers processors with a handful of chiplets inside. The company has found that just by stacking a memory chip on top of its CPU -- the chip that does the bulk of the non-graphics calculations in a computer -- it was able to significantly increase the speed of its systems.

While chiplet-based megachips may currently be few in number, the trend toward making them is accelerating, says Marc Swinnen, director of product marketing at Ansys, a company that builds physical simulation software that is used widely across the microchip design industry. The number of projects by Ansys customers involving stacked chiplets has increased 20-fold since 2019, when it was in the low single digits, says a company spokesman. (For perspective, the total number of such chip-design projects under way in the world at any time is estimated to be in the hundreds.)

A major driver of the industrywide interest in this technology is the desire by a growing array of companies -- including Amazon, Google, Microsoft, Tesla and others -- to create their own, ever-more-powerful microchips to run everything from cloud services and smartphones to gaming consoles and vehicles.

"There are now entire divisions of major companies whose very business proposition rests on the quality of their silicon," says Mr. Swinnen.

Also driving interest in megachips is the rapacious demands that artificial intelligence and machine learning systems place on existing hardware, says UCLA's Dr. Iyer. While some have responded to this need by building truly gigantic microchips the old-fashioned way, others, including Dr. Iyer's team, are working on AI-focused megachips composed of chiplets." [1]

In short: Chip sprawl is just getting started.

1. EXCHANGE --- Keywords: The Age of the Megachip Is Dawning --- Mini-metropolises of silicon are coming to our devices
Mims, Christopher. 
Wall Street Journal, Eastern edition; New York, N.Y. [New York, N.Y]. 30 July 2022: B.4.