Apple has done pretty well for itself selling phones, tablets, computers, and smartwatches — but it didn’t invent any of those things. The company prefers to leave the first drafts of new technologies to other companies, watching them make design mistakes, fumble, and flop.
Only after a few years does Apple storm onto the scene with its own better designed, more completely conceived entry — and then run away with the market. Usually (*cough* HomePods *cough*).
Never has Apple followed that playbook more closely than with its Vision Pro augmented-reality goggles, its first major new product in years, which went on sale this week for a throat-clenching $3,500.
Most headsets cost far less; Meta’s Quest 3, for example, goes for $500. On the other hand, most of the less expensive headsets have screens that are so laggy and low-resolution that they can nauseate you.
Furthermore, most of those products are virtual-reality headsets, meaning that they block out the world around you, including friends and family. VR is supremely isolating.
Apple’s first big idea with the Vision Pro is technical superiority; the tech inside is certifiably insane. The company racked up 5,000 patents building this thing. It’s got 12 cameras, two infrared beams and receivers, six microphones, and a Lidar scanner.
Above all, it contains ridiculously sharp screens. One for each eye, each with the resolution of a 4K television. (Packing so many pixels into so little space means that each pixel is only 7.5 microns across. That’s about the size of a red blood cell.)
Those screens make possible Apple’s second big idea: transparency. When the goggles are on your face, they’re not blindfolds; you still see the room (or office, or plane) around you.
Now, you’re not actually looking directly at the room around you. It’s a live video feed from tiny cameras on the headset. But the eye screens are so sharp, the illusion of direct vision is generally flawless.
The point of all this is to run software, play movies and games, review photos, and so on. Those software windows float as huge, clean, lovely, vertical planes within your actual space.
Apple CEO Tim Cook has championed this project for years despite delays, cost overruns, and internal bickering about its prospects. And so, maybe because he wanted to leave nothing to chance, this first version feels more complete and polished than any earlier 1.0 Apple product. (The original iPhone, for example, didn’t have an App Store, video recording, a front camera, or any way to copy and paste.)
But assessing it is really a two-part job. There’s the concept, which Apple calls spatial computing, and then there’s the execution. Turns out that with this first version, Apple did better with the former than the latter.
I’ll give you the bad news first: The Vision Pro is heavy. At just over 1.4 pounds, it weighs about the same as nine “C” alkaline batteries.
That’s not even counting its own battery, which is an external .75-pound silvery brick on a three-foot cable. (It powers the headset for about two hours. You can also plug the battery into the wall, in which case the headset runs forever.)
Apple certainly tried to make all that weight comfortable. There’s a lot of aluminum and carbon fiber involved. You get two different headstraps: one that wraps only around the back of your skull and another with a band over the top. (Hope you didn’t just get your hair done.)
You can’t buy the Vision Pro without getting your head measured, either with an iPhone scan or in an Apple Store, so that you’re getting the proper “light seal” separator. If you wear glasses, you’ll also need a pair of magnetically in-snapping Zeiss correction lenses, which cost $100 if you use drugstore reading glasses or $150 for prescriptions. The box comes with two different face-pad strips, too.
Even after all that, though, it can take a lot of fussing to avoid a 1.4-pound ache on your forehead or nose bridge.
Once you have it all assembled, you go through what may be the most beautiful setup wizard ever designed. The exterior cameras scan your hands, eyes, and face, accompanied by a gorgeous soundscape of musical tones. (The speakers are excellent. They’re right next to your ears but don’t cover them, which means that whoever’s next to you on the train will hear leaked sound. Apple brightly suggests that you wear AirPods in that situation.)
What You See
There are a couple of fascinating footnotes to that business of the Vision Pro offering a simulated see-through to the world around you.
First, if you prefer to be isolated from the world, you can turn a little knob above your right eye. It controls the size of a “curtain” of blackness (or of a scenic backdrop).
But even in full hide-the-room mode, if somebody approaches you, something surprising happens: A cloudy-edged hole opens up in that curtain so you can see them. Yes, even in full blacked-out mode, you are still not isolated.
Where the Vision Pro’s tech goes from impressive to absolutely bonkers is in the feature that was widely discussed (and even mocked) after Apple debuted the product in September: the video screen on the outside of the headset and corresponding cameras on the inside that show your face.
The effect is a bit ghostly and blurry, not as sharp as it appears in Apple’s videos. But if someone’s trying to talk to you, it’s better than talking to a plastic faceplate.
What other people are actually seeing on the outside of the goggles is not just a photo image, but rather a 3-D rendering of your head. Apple calls it your Persona — in essence, it’s a deep fake of you.
To create it, during the initial setup, you’re invited to take the goggles off and turn them to face you. From the speakers, a woman’s voice prompts you to tilt your head left, right, up, and down; to smile, with and without showing teeth; and to raise your eyebrows and to close your eyes. From that, the software builds your 3-D head model.
Your Persona also appears to other people in video chats while you’re in the headset — FaceTime, Zoom, and so on. It does a great job of representing your actual, real-time facial expressions. Nobody, however, will mistake your CGI Persona as an actual video of the actual you. It’s more like “way down in uncanny valley” (my wife) or “a nightmare version of you” (my son).
Apple says that Personas are still in beta testing. Let’s hope so.
What Apple Vision Pro Does Well
So you’ve spent all the money, fussed with the straps, and built your 3-D avatar. What, you might be wondering, can you do with this thing?
There is no “killer app.” (Frankly, I can’t stand when reviewers say, “There is no killer app.” When was the last time anything had a killer app? Probably 1979, when the term was coined for VisiCalc as the sole reason to buy the Apple II. I mean, what’s the killer app for the iPhone? The iPad? Your oven? These days, an invention doesn’t become successful because of one killer product; it should enhance all kinds of aspects of your life.)
The Vision Pro’s OS is a lot like an iPhone’s or iPad’s. There’s a Control Center for changing settings quickly, for example, and a home screen that shows all your app icons. (To see it, you press that button above your right eye.) Out of the box, you get the usual iOS apps — Safari, Mail, Music, TV, Photos, Messages, and so on — but you can also run any iPhone or iPad app, as well as 600 apps (so far) written or updated just for the Vision Pro.
(Wanna know what’s super weird? Only what you’re looking at is razor-sharp. To save processing and power, the Vision Pro slightly blurs the rest of the screen — but because, by definition, you’re not looking there, you don’t notice it. The blurring does show up in captured screenshots and videos, though, like the ones in this article.)
There’s no mouse. Instead, the Vision Pro uses an incredibly precise eye-tracking system; your gaze is the cursor. To click something, you look at it and then lightly tap your thumb and forefinger together. Somehow, the headset’s external cameras can see that tap no matter where you’re doing it. On your leg. On the couch. With your arms crossed. It takes about four seconds to master, and it works every time.
You can also scroll things by tugging in the direction you want, as though on an invisible window-shade cord, and you zoom into things by tugging their corners diagonally with both hands.
Once you learn those moves, here are some of the stunts the Vision Pro makes possible:
Set up an infinite number of massive monitors.
You can position any app’s window anywhere — to your right or left, on the floor, over there — and adjust its size, position, and distance.
No matter how your head moves or where you walk, they remain rooted where you place them, as solidly and physically as paintings or chandeliers. You can slap a big photo onto your bathroom wall, hang a timer “card” over a boiling pot on your stove, and then lie on your couch to watch a movie on the living-room ceiling — and everything will still be in precisely those spots when you return. Certifiably insane.
Work on your Mac.
When you look at your laptop, a Connect button appears. If you “click” that button, the Mac’s screen goes black — and its image leaps upward into space. You are now free to make it huge. Turn it into a Jumbotron, for all the Vision cares. You can use your laptop’s trackpad and keyboard as usual. (For Vision Pro apps, you can use a wireless keyboard, or you can type in midair on an on-screen keyboard.)
Several big software companies have already Vision-ified their apps, including Box, Webex, Zoom, Slack, Notion, Todoist, and Microsoft with Word, Excel, PowerPoint, Outlook, OneNote, Loop, and Teams. There aren’t apps for Netflix or YouTube yet, although of course you can play their videos in the web browser.
I do a lot of video editing in Final Cut Pro, and that means a lot of side-to-side scrolling. Let me tell you: When I realized that I could do this editing on the equivalent of a 200-inch monitor — while actually in a cramped airplane seat — I just about spit out my Sprite.
In one preview of the Vision Pro from September, a critic wrote, “Working on documents is the weakest argument for an AR/VR headset. Movies, concerts, games, video calls, and live sports broadcasts — I get it. Gigantic, life-size representations make a thrilling difference. But Microsoft Word and Notes? Seems like a nice big monitor would work fine.”
Okay, that was me, and I was kind of wrong.
True, everything from your Mac is confined to one big window. But you can also open as many Vision Pro apps as you want and park their windows all around you. It’s genuinely useful to have Messages just to the left of the Word window, your calendar off to the right, and maybe Safari on the floor. No hunting, no window shuffling.
Look at things really, really big.
You’ve never truly seen your iPhone panorama photos until you see them with the goggles on. They’re as tall as the room, and they wrap entirely around you. “Immersive” barely covers it.
And movies — oh my God. I don’t care how tiny your apartment is, how cramped the back seat of your Uber: You can watch movies at Imax size. Bright, pristine visuals, pounding “spatial audio” pumped right into your ear sockets. 3-D movies are especially convincing.
(Incidentally: When you’re lying on your back, the weight issue seems to go away. Movie buffs, do with that info as you will.)
Share your view.
Here’s yet another way Apple promotes transparency: You can let other people see what you’re seeing, on a nearby iPhone or Mac, or over FaceTime.
In fact, if you’re in Taylor Swift’s tax bracket and own multiple Vision Pro headsets, you can even watch a movie or TV show on all of them simultaneously, perfectly in sync. Once again, Apple wants to blast away at the notion that headsets are isolating.
Pictures and videos.
A shutter button sits above your left eye. The photos and videos it captures are in 3-D; provided that the subject is close to you, the depth effect is convincing and super cool.
The iPhone 15 Pro can also capture these so-called spatial photos and videos — but of course you can’t view the depth effect until they show up on the Vision Pro.
3-D apps and games.
The headset comes with a dinosaur-encounter app, in which a delicate butterfly flutters in and lands on your finger. Then, as it leaves, the walls of your room open up to reveal a prehistoric landscape. At this point, a realistic CGI raptor lumbers by, notices you, tracks you as you walk around — and then steps out of the screen into the room with you. It’s a one-trick pony, this app, but a hell of a trick.
Super Fruit Ninja is a true augmented-reality game — that is, the fruits and characters are there in the room with you, and you use your hands to grab and chop them. (Any iPad game works great in Vision Pro; you can even use a physical game controller, from an Xbox or whatever, in games where eye tracking would be a little clumsy.)
What Needs Work
Most people would find all of this thrilling and new, but the Vision Pro also has its flaws.
Now, expensive is not the same as overpriced. The amount of new, custom, advanced technology in this thing is preposterous, and it truly is better than what’s come before. But that doesn’t mean most people can afford it.
For me, the biggest showstopper will be the weight of this thing. If you’re using the standard headband — the elastic one that goes only around the back of your head — the front-loaded design of the Vision Pro eventually becomes uncomfortable. It just does. It’s just too much pressure on your forehead, cheekbones, and nose.
The other headstrap in the box, known as the Dual Knit Band, is the one with a strap for the top of your head. There’s still a lot of fussing and adjusting, and now some of the weight is on your skull, but at least it’s not all on your face.
There’s no on-off switch.
The only way to turn off the headset is to disconnect the power cord or to tell Siri, “Turn off Vision Pro.”
The clarity drops in low light.
In low light, the video from the outside world shows motion blur when you turn your head, and even the resolution seems to drop.
The carrying case is ludicrous.
It’s this massive pillow-looking thing that seems to be six times bigger than necessary. It costs $200, and it’s white. How long will a white carrying case stay clean?
Other reported issues.
Some early buyers are also complaining about a few things I didn’t mind:
The eye tracking is frustrating.
On Reddit, some people say they’re having trouble with the eye tracking. For me, though, the eye tracking is one of the highlights. I can easily click tiny checkboxes, for example. (My guess is that at least some of the critics are sharing the headset with friends who haven’t been through the setup and calibration process.)
The field of view is limited.
“Like looking through binoculars,” said one reviewer. It’s not like you can see 180 degrees without turning your head, but the view has never struck me as limited.
There’s no 3-D porn.
Apparently, that’s true.
Concept vs. Execution
As it is, the 2024 Vision Pro is a prize for early adopters and Apple superfans, and maybe a fun purchase for the well-heeled curious who want to take their first steps into headset computing. But with its current issues of price, weight, and comfort, the general public won’t beat a path to the door of Vision Pro 1.0.
But that doesn’t mean that the Vision Pro is a flop. As an experiment to prove that goggle computing can work, it’s a home run. It establishes a simple, clear, precise system of manipulating virtual objects — that doesn’t require a plastic controller grip in each hand or location beacons on posts around your room. And the superiority of its eye screens proves that people don’t have to get seasick from using headsets.
There will, of course, be another Vision Pro, another version after that, and another one after that. Each version will chip away at the price and weight problems, and each will endear itself to a larger audience.
At half the price and half the weight, the advantages of the Vision Pro concept will be too hard to ignore: vast, bright, crisp screens — for your movies and your software — no matter how cramped, dim, or grubby your space in the real world.