The 'Killer Apps' That Will Make Apple's Headset a Winner --- Augmented and virtual reality have mostly been dead ends so far. Apple's vision could succeed where others have failed.

"Explaining the inner workings of a nuclear-powered octocopter that will someday explore Jupiter's moon Titan is a tall order for even the most skilled teacher.

Luckily, when I recently paid a visit to the lab where it's being designed and tested, an engineer tasked with building the Dragonfly probe for NASA pulled out a pedagogical aid sufficient for the task. It was a HoloLens 2, the augmented-reality headset made by Microsoft.

Each of us wore a HoloLens headset that projected a realistic, shared, three-dimensional image of the Dragonfly in front of our eyes -- a sort of shared hallucination. We were able to walk around the space probe, my guide peeling back its layers and pointing to individual components.

That experience was remarkable precisely because it couldn't have been more compelling and edifying in any medium other than augmented reality -- the name for tech that imposes digital images onto our view of the real world. In the not-too-distant future, that kind of experience could be utterly mundane. Just as personal computers and then smartphones were once exotic and now both teach and distract us daily, augmented reality is coming for how we interact with the digital and physical worlds.

Apple's forthcoming headset, which the company is expected to reveal at its developers' conference starting June 5, is likely to accelerate the adoption of augmented reality -- and the broader phenomenon of "mixed reality," which encompasses full-immersion virtual-reality headsets that completely occlude a person's vision.

Apple has a chance to succeed where others have failed. Microsoft, a pioneer of augmented reality and maker of the impressive, but limited, headset I wore that day, is a good example of why it hasn't caught on yet. Despite an early lead in the field -- the first HoloLens shipped in 2016 -- the company largely failed to make a compelling case for its technology, and has seen an exodus of talent from the team responsible for it. A similar fate befell Google Glass, which Google stopped selling earlier this year.

The key to Apple's success -- or failure -- will be its ability to mobilize not just its fanatical, and growing, fan base, but also an army of developers, to make apps and services for its headset the way they have for its other devices.

Motivating those developers will require that Apple demonstrate what has so far been lacking for mixed reality: a blueprint for "killer apps" that will be so compelling that early adopters plunk down for this and future Apple headsets. (No small ask given that the device is expected to cost $3,000 or so.)

My experiences with many of the state-of-the-art AR and VR systems that have come out so far suggest that these killer apps will do things that the flat screens of our current devices struggle with. Most important and transformative: They will let us interact, intuitively, with content in three dimensions.

The first major augmented reality hit was a 2016 game called Pokemon Go, in which players wander the real world with their phones out, collecting pet monsters. It showed that there can be an appetite for gaming using this technology, but the fact that in all the years since, there has yet to be another AR game that matched its success suggests future AR games might be quite different from walking through your neighborhood trying to "catch 'em all."

Something that doesn't require gamers to actually go outside might do better. In the first six weeks since its launch in March, the Sony PlayStation VR2 headset sold around 600,000 units, the company revealed in a recent presentation to investors. That's a tiny fraction of the number of iPhones sold in the same period, but it indicates that games are a place where there is genuine interest in mixed-reality devices. There are countless examples of AR games already available in Apple's app store that can be played on iPhones, evidence of Apple's yearslong commitment to educating both its users and developers about how this technology works. While some, like the Lego AR Studio, are well-designed, almost all of them still feel like tech demos, not fully formed products.

It's clear that for AR games to take off, a fully immersive, headset-based experience that frees the player's hands to interact with the world -- like Sony's VR system -- will be required.

Beyond 3-D interfaces for apps we already use on our existing devices, mixed-reality systems have the potential to help us better interact with information that is already inherently three dimensional.

For example, when we navigate with a map, we have to translate from the real world to the map, and back. This is tolerable for people who have grown accustomed to it, but there is a more intuitive way: What if we could simply overlay arrows telling us which way to go on the real world? Since 2019, Google Maps Live View has made this possible on our phones, and it's indisputably handy when trying to walk around an unfamiliar city.

Making augmented-reality navigation work on a headset, versus a phone, requires that the headset be something comfortable, safe and stylish enough that people aren't afraid to wear it on their face, in public. Accomplishing that while also giving a device a powerful enough processor and a good enough display is a tall order, says Ari Grobman, chief executive of Lumus. His company has spent 22 years developing components that he says will someday enable just such a pair of smart glasses.

Apple's forthcoming headset seems to be the opposite of Grobman's vision of smart glasses, apparently more like a traditional VR headset, and resembles ski goggles. Even if users aren't at first comfortable walking around outside in Apple's headset, there is plenty of inherently 3-D information worth consuming on such a device, indoors.

Andrew Garnar-Wortzel, a principal at architecture firm Gensler, has used systems like The Wild, which allows architects and their clients to experience an in-progress building in virtual reality. Doing so can enhance the process of designing and tweaking such a building, by making it more real than simply looking at blueprints or renderings on a flat screen. "While in the middle of designing with clients, we could meet in the space we were designing," says Garnar-Wortzel.

For now, Apple appears to be opting for a more maximalist approach than many of its competitors in augmented reality, releasing a headset that resembles today's VR headsets -- such as those made by Meta and Sony. This allows for more computing power in the headset, and more capabilities, but such a design sacrifices stylishness, comfort and wearability, and is a far cry from lightweight "smart glasses."

In some ways, this form factor could be a mistake.

Tamir Berliner is a former executive at augmented-reality startup Magic Leap, and now the chief executive of Sightful, a company that just unveiled a laptop whose only display is a pair of lightweight augmented-reality glasses.

One of the first things he discovered when the company began testing its product with focus groups was that if he gave testers VR-style displays, they automatically excluded about half the people who might otherwise use the company's device. People with long hair generally don't like putting bulky straps on their heads, says Berliner. And anyone wearing makeup is disinclined to put on something that resembles ski goggles, which will press into their face.

All the smarts and capabilities Apple can cram into its headset won't matter if people don't feel comfortable wearing it. Apple certainly has to start somewhere, in terms of getting developers accustomed to what will likely be just the first device of many, and getting consumers excited about its potential.

What remains to be seen is whether the capabilities of this device, and the potential of 3-D content and 3-D interfaces, will overcome the issues that have plagued so many other tech giants that have tried and failed to make face-worn computers go mainstream." [1]

1. EXCHANGE --- Keywords: The 'Killer Apps' That Will Make Apple's Headset a Winner --- Augmented and virtual reality have mostly been dead ends so far. Apple's vision could succeed where others have failed. Mims, Christopher. 
Wall Street Journal, Eastern edition; New York, N.Y. [New York, N.Y]. 27 May 2023: B.5.