The VR and AR arms race

You probably don’t remember the Humane Pin, despite its dominating the tech-news cycle a few months ago.

It’s an elegant AI-powered square that sticks to your lapel and can send messages, search for information and tell you about what you’re seeing, all through voice commands. The Humane company raised more than $230 million in venture backing, and its rollout included a runway appearance at Paris Fashion Week, a TED talk and a chic announcement video where the pin repeatedly misinformed its user. According to the pin, almonds have far more protein than they do, and April’s solar eclipse would have been best viewed from Australia, where it wasn’t even visible. Also, there’s no way to use the pin other than with voice control. So don’t sext on it or ask private medical questions in public; everybody else will hear the responses. It starts at $699, plus a mandatory $24 per month subscription.

Humane’s Pin is a beautiful prototype for the post-smartphone future, but that future isn’t here yet. For now, it’s better to stick with your phone. I was thinking about this as I tested Apple’s new $3,500 headset, the Vision Pro.

Apple framed the February release of the Vision Pro as the next iPhone moment, the debut of a beautiful device that will revolutionize the way we work and live, the first “spatial computer.” Throw out your ugly monitors and chunky tower PCs! As I popped on the Vision Pro, digital widgets and windows floated before me in midair, casting soft shadows into the room; they were controlled as though by thought, by looking at them and tapping my fingers. It was digital telepathy — and I could open as many windows as I wanted, wherever I pleased. With the scroll of its “reality dial,” the world faded away and I was at Joshua Tree, hearing the sounds of the natural world.

Trying it felt like the future. It made me excited about tech — and I wasn’t alone. When the headset released, Twitter was flooded with people pretending to use them as they drove, or sat on the subway, or stepped out of Cybertrucks looking like jackasses. YouTube was filled with viral content of creators trying it out.

But in the months since, you probably haven’t thought much about the Vision Pro. You didn’t buy one and you don’t feel as though you’re missing out on the revolution. That’s not surprising. Despite its revolutionary aura, this idea is nothing new. Inspired by Star Trek, Snow Crash and Ready Player One, generations of tech nerds have tried to build virtual reality (VR) headsets that bring us into digital worlds, and “augmented reality” (AR) ones which project digital elements onto the real one. Early versions were loud, heavy, low-resolution and intensely nauseating (literally: they caused motion sickness in many users). But the nerds never gave up. The promise was too tantalizing.

Imagine traveling to distant lands, exploring space, touring atomic particles, hanging out with distant loved ones, building worlds that Newton says couldn’t exist. And all just by putting on a headset. You could tour the temples of Angkor Wat from your living room, work on any number of digital screens, and if you don’t love the view from your flat, change it instantly with the tap of a button. For futurists and tinkerers, it felt too incredible to leave in the realm of fantasy.

Over the past decade, development of virtual-reality tech has grown exponentially. What used to be a tech-show gimmick is now used to assist soldiers in combat, perform surgery and design cars. Having bought VR-startup Oculus for $2 billion in 2014, Mark Zuckerberg changed his company’s name in 2021, from “Facebook” to “Meta,” to “reflect its focus on building the metaverse” (an internet built around three-dimensional digital spaces). The future is coming.

But this push isn’t born of sheer utopian spirit. Though Silicon Valley runs on the idea that the future is what we choose to make it, it only exists because of the companion belief that you can get unfathomably rich doing so. And virtual worlds contain very real gold mines.

As it stands, if you build a smartphone app, Apple and Google take a 30 percent cut of every sale, and 15 percent of all annual subscriptions, and if you tell users to subscribe on the web to get around these fees, you might see your app removed from stores. This is frustrating to every company that isn’t Apple or Google and has led to both being pursued on antitrust charges and facing steep fines from the European Union. But if Meta could make headsets for the next universal computing platform and replace the smartphone, they’d escape the fees. Even better, if they move fast enough, they could become the next app-store tax collectors themselves. They wouldn’t be paying 30 percent. They’d be taking it.

As though this weren’t enough motivation, headsets are a massive boon for online advertising, of which Meta — uncoincidentally — is the second-largest broker. Think about how accurate TikTok and Instagram’s ad recommendations are now. Based merely on phones’ location, web activity and screen time, they can target ads with such precision that many people falsely think Facebook is recording their conversations. Imagine how much more accurate this would be if everyone’s devices tracked their hand, face, eye and body movements, along with fitness levels, usage times, location information and a live camera feed of their surroundings. It’s not incidental that TikTok owner ByteDance offers a VR headset, the Pico 4, nor that the Chinese Communist Party released a five-year plan in November 2021 that aimed to sell 25 million VR headsets annually by 2026.

In the years leading up to the launch of the Vision Pro, Apple reframed itself as a privacy-centric company. It put up Vegas billboards that read “what happens on your iPhone, stays on your iPhone.” In 2021, CEO Tim Cook went on a PR tour attacking the advertising model of their main VR competitor, Meta. That year, Apple rolled out a new “ad tracking transparency” feature, which ostensibly gave users more control over their data, but in practice gave Apple a giant advantage on advertising based on iPhone user data.

The move cost Meta roughly $10 billion in ad revenue over the following year, a period in which Apple grew its own ad network to $4 billion annual revenue. Apple’s target is reportedly to reach $10 billion per year in ad revenue by 2026. The new Vision Pro, a device covered in cameras, is ready to extract advertising information from their most high-end customers.

The Vision Pro is beautiful. It’s dense, chic, made of cold-to-the-touch glass and metal — it’s unquestionably special. I can’t remember holding a consumer tech product — something that anybody who lives in the US can walk into a store and buy — that felt so truly futurist.

But it’s not unique, and it’s certainly not the best or most advanced VR headset on sale. It’s heavy, uncomfortable to wear for more than half an hour, only properly configured for a single user, relatively fragile (and susceptible to cracking along the outside glass) and it only has two hours of battery life, meaning you’ll be plugged in most of the time. Its battery hangs from a long cord, to be stuffed in your pocket like a 1940s hearing aid. (Apple tried to hide this less-than-sleek aspect in their pre-launch photos.)

Most importantly, it’s not clear exactly what the Vision Pro is for. What do you do with it after spending the $3,500? Apple says you can video call, watch movies and work in it, but the “persona” avatar for video calling is an uncanny digital waxwork of your face, so I’d rather call on my phone, just as I’d rather watch a movie on a big TV screen, beside friends, and work on my Lenovo Ultrawide monitor, which is far uglier and not infinitely adjustable but doesn’t hurt my head. There are cool renders for potential apps, particularly for viewing live sports — the field or track hovers in front of you, and you’re surrounded by live TV feeds, showing the action from all possible angles. But they aren’t real yet, and it will be years before they are. And sure, it has a few games, but they’re mostly subpar copies of other popular VR titles. Even if you could get great games on it, do you really want to get sweat on a $3,500 computer, or risk damaging it?

As it stands, most Vision Pro exclusive apps are either iPad apps with a pleasant but soulless transparent look, or demos. You try them once, go “huh, that’s kind of cool,” and never use them again. By comparison, the Bigscreen Beyond, billed as the world’s smallest VR headset, has a better screen, a massive game library through SteamVR, a far lighter package — it weighs only a fifth as much as the Vision Pro — and it’s $2,500 cheaper. Plus, because it’s so light and has a great screen, you can wear it long enough to watch movies on it.

But then again, the reason it doesn’t feel like anything else is that it was made with a completely different philosophy. The Vision Pro is not like the other girls, because Apple isn’t like other companies. It’s too big to compete with; if it did a conventional VR headset it wouldn’t titillate investors and pump the stock further. But a devkit of the “spatial computer?” A premium product, for their most dedicated fans, that heralds the next era of computing? That just might. With iPhone sales gradually declining, Apple wants to create and dominate the next universal computing platform, to be the leader in a future multibillion-dollar “spatial computing” market, where small AR headsets replace phones as our primary computers.

That future is far away, and the Vision Pro pays more attention to previewing it than to refining its current usefulness. Still, it’s a pretty appealing future.

The year is 2040. California is full of electric cars. Trump claims the last election was stolen. Avocado toast costs $80 in Brooklyn. NFTs still aren’t a thing. Sam Bankman-Fried is in prison. CNN claims America is on the road to fascism. Little has changed.

But instead of a phone or laptop, you have a pair of smart glasses. On a single device, you work, make phone calls, listen to music, search the web, record memories, send messages, watch movies and do any number of other tasks. Your specs look like the Persols James Dean wore, save for their bulky earpieces, which contain speakers, cameras, microphones, batteries and processors, and you wear them almost constantly.

When you want to go to a new café, you ask your onboard AI for directions, which appear on the pavement before you; as you walk, you swipe and scroll through the emails in your floating, semi-translucent inbox, controlling it with slight gestures of your hands. When you get to your table, you whip out a keyboard and all your work windows and tabs appear exactly as you left them yesterday. And with a tap, or the turn of a dial, all background distractions fade away and you’re in the quiet virtual workspace of your choice. (You may or may not have to ask a human being for your coffee.)

We’ll be lucky if we can pack all that computing power, thermal capacity, battery life and screen quality into such a small form by 2040; to truly replace phones and laptops, we’ll need to — imagine a cooling fan next to your temple. Put a Vision Pro side by side with the prescient Google Glass and you can see how far we have to go. But there are glimpses of it already. Despite ads showing people watch movies and search the web, one of the best demonstrations of augmented reality is a video of a man repairing his car while wearing a Quest 3, scrolling through instruction manuals and watching a how-to video that floats in midair.

Perhaps the best idea of how it will all work can be gleaned from current smart glasses, comparatively basic devices that simply try to shove as much computer functionality onto your face as possible. There are several coming along: Google has demonstrated a pair that live-translates your conversations, and the startup Brilliant Labs promises a pair of open-source AI “Frame” smart glasses; neither has been released.

But already I’ve listened to music, made phone calls, sent WhatsApp messages, videoed passing cars and taken photos of loved ones using Meta’s fantastic AI-equipped RayBan Wayfarers, and I don’t go for a walk without them. Unfortunately, they don’t have a screen. XReal’s Air 2 smart glasses do; by plugging them into your phone or laptop, you can mirror their screens onto the glasses — but you won’t want to. The image is fixed, small, blurry, incompatible with many phones and — in my experience — deeply nauseating. My headache lasted for hours after I took them off. I promptly returned them to Amazon.

These VR headsets and smart glasses are both heading toward those imagined 2040 sunnies, getting smaller and more capable respectively with each new generation.

Today’s Vision Pro, on the other hand, is an overdeveloped tool for developers to rig “spatial” apps on. It allows rich tech enthusiasts have a taste of what the imagined future will feel like. It’s impractical and most people shouldn’t buy one, but — in short bursts — it’s an exciting look at the world to come. The software is intuitive and elegant, its eye-tracking feels like telepathy and watching 3D “spatial videos” is like being back in the moment they were captured. In the decades to come, when you watch spatial videos of loved ones, it’ll allow you to feel like you can still be with them even after they’re gone.

The long and short of it is that the Vision Pro feels like what we think the future will be. But, as with Humane’s Pin, that future is way undercooked, or still being mixed and measured, or missing crucial ingredients. It’s not ready yet. Despite what some credulous journalists say, you never feel like you’re looking through it. You’re always conscious that you’re boxed in, staring at screens, with weight pressed against your eyes. No matter what strap I used, I couldn’t get comfortable, and though I sometimes believed my augmented vision, that only made the breaks more jarring — the world got grainy when the sun went down, lights flickered, digital objects seemed far brighter and more colorful than physical ones. When I moved around and objects weren’t as far or close as they seemed, or I tried to check my phone’s notifications and the screen was a blurry glow, I had a queasy reminder of the unreality of what I saw. In a fully virtual world, you buy into a consistent fiction; augmented reality often reminds you not to believe your lying eyes.

All these issues have to be buffered out before people will consider using headsets as their daily, primary computers. And that’s before you get to the moral and philosophical problem of trusting the companies that make them. Meta, Apple and Google have all faced huge controversies for their choices in content moderation, curation and generation — from Facebook’s pushing of edgy news to Apple’s refusal to allow anything adult-related and Google’s hammering forced diversity into their image-generation AI. That will only seem more pronounced with computers you wear. Would you trust them to influence how you literally see the world? Though it’s amusing, the difficulty that Vision Pro users face when trying to play VR porn is a stark reminder that Apple isn’t a neutral arbiter, letting you use your device as you please. They have a look and feel they want to maintain — a version of reality they want to curate — and you’d better like it. Zuckerberg has tried to frame Meta’s vision for VR as the open, free alternative to Apple’s closed one, but you can’t customize the Quest’s software, or upgrade its components, or even download programs that don’t come from their app store.

When I received a review sample of Meta’s bestselling Quest 3 in December, I thought it was a fun toy, a gadget to buy at Christmas instead of a Nintendo Switch. Its interface is generations behind Apple’s, its response to hand gestures so unreliable that I almost always use the controllers, and it’s even worse as a workplace or TV replacement than the Vision Pro.

Similarly, though Meta has repeatedly mastered messaging and social media with Facebook, Messenger, WhatsApp and Instagram, its Horizon Worlds “metaverse” is a terrible, ugly cartoon environment visited by only a few oddballs. I have no desire to spend time there. But it has great screens, a capable processor and a vast library of games, and its cut-down resolution, controller complexity and tracking systems let it hit an accessible $499 price point. It’s far from the immersive metaverse portal that Zuckerberg dreams of, but Meta has prioritized what its customers want — a VR headset that’s fun to use. After its release, Meta’s “Reality Labs” pulled in over $1 billion in Q4 2023, and sales will only look better when they release an even cheaper Quest later this year.

Where the Vision Pro has a smattering of briefly entertaining demos, the Quest has an endless selection of games to choose from. There are licensed Ghostbuster and Wallace & Gromit adventures, Meta’s premium fantasy roleplaying game, Asgarth’s Wrath II, the exploration-learning game Out of Scale, the swinging shooter Swarm 2, classic first-person shooters like VAIL, Pavlov Shack and Ghosts of Tabor, and climbing sims and creation tools and augmented piano games that teach you how to play your actual piano. And it also supports every PC VR game through Steam Link, which allows you to stream more computer-intensive titles, like Half Life: Alyx, from a gaming computer. These games aren’t just “good for a VR game”: Humanity, Super Hot VR and UNDERDOGS are fantastic games by any standard, and you could burn hundreds of hours in them with little effort.

If that were all, the Quest 3 would be like my gaming laptop, great but underutilized, because after my workday I don’t want to waste those hundreds of hours on another screen. Nevertheless, for the past several months, I’ve used the Quest 3 for more than an hour a night, almost every night. For me, the Quest 3 isn’t competing with a MacBook or Nintendo Switch or phone or tablet; nor is it a cinema, office, masturbatorium or metaverse social scene. It’s a Peloton competitor.

Not having kids, a commute or a work-life balance, I usually wrap up work around 10:30 at night, and at that point, it’s hard to convince myself to stop and work out. But throwing on the Quest and playing some motion games feels more like playing than working out — though I’d never really believed in VR exercise, which seemed like a gimmick. But with Meta’s silicon face shield and BOBOVR’s M3 Pro head strap, you have the best value-for-money workout machine on the market, slicing boxes in Beatsaber or shooting enemies in Pistolwhip. Beating the life out of the air with Les Mills boxing classes can be a real workout. Within twenty minutes, I’m sweating; by the end of the hour, I’m panting and soaked through, like I’ve gone for a run.

I’m glad the Vision Pro exists. It’s the first truly ambitious thing Apple has released since the twelve-inch MacBook, and will bring more attention to the VR space, accelerate the building of compelling spatial computing and virtual reality, and prod its competitors to make more advanced premium headsets — a Meta Quest Pro 2, or the Sol Reader for distraction-free reading, or the “Virtual Eye,” focused squarely on “fitness gaming.”

Apple doesn’t need to make super-smart glasses to have a desirable product. In four generations or so, a lighter, more powerful, more comfortable version of the Vision Pro should be a meaningful competitor to the static, monitor-based computing of today, and I really look forward to that; just as I’m excited for metaverse virtual worlds to go from lame test software to gripping, life-like experiences.

In the meantime, I’m back to my VR gym. The future is coming; and I’m going to be ripped when it gets here.

This article was originally published in The Spectator’s May 2024 World edition.