At F8 today, Mark Zuckerberg stood onstage and paused his presentation for a moment. “If you take one thing away with you from today, this is it: We’re making the camera the first augmented-reality platform.”
That’s a, uh, bold claim. There’s a company — a self-described camera company that Facebook has taken an awful lot of ideas from — that might claim the same thing: Snapchat (or, as the company styles itself now, Snap, Inc.). In the company’s IPO SEC filing, it wrote: “In the way that the flashing cursor became the starting point for most products on desktop computers, we believe that the camera screen will be the starting point for most products on smartphones.”
That point was driven home when Snapchat released World Lenses earlier today, an impressive bit of tech that uses some of the same tech used for the cool face filters in Snapchat to place 3-D objects in the real world, using the rear-facing camera in a way that feels much more anchored in reality than the stickers previously available. In many ways, Snapchat has allowed for augmented reality for over a year now — what are face masks and filters but overlaying the digital on top of the physical?
It’s hard not to get the feeling that someone at Snapchat knew what was coming at F8 today, because Zuckerberg quickly ran through what he meant by “making the camera the first augmented-reality platform,” and it’s a vision that makes Snapchat’s World Lenses seem primitive in comparison. As Zuckerberg laid it out, your smartphone camera will act as a window into a world filled with digital information, overlaid on top of the physical world. It’s a compelling vision — even if it’s one that may be years away.
First up were face masks and photo filters, but instead of the few dozen that Snapchat designs in-house and offers a handful at a time, Facebook says it will open it up to developers and make available thousands of face masks and filters. Unlike Snapchat, which creates its own filters, working closely with brands while making sure that the Snapchat vision remains on point, Facebook seems willing to let thousands of devs, designers, and artists “of all different kinds of cultures and background and styles” into the pool. It’ll make for a lot more variety (though, of course, a decline in average quality).
Up next was something more impressive: A scene at a breakfast table, in which the user was able to make text rise up out of the tabletop and sharks swim in a circle around a cereal bowl, all of it with the digital animation perfectly in sync with the physical world; the sharks rose up and disappeared back into the tabletop, and disappeared from view when they went behind the rim of the cereal bowl. The only AR I’ve seen able to replicate this level of physical-and-digital interaction is Microsoft’s HoloLens, which currently costs about $5,000.
Zuckerberg also showed off the ability to take a still photo and map it out onto a 3-D space. This means, you can not only pan around the photo, but do things like change the lighting in the room, fill it up with digital water, or fill the room with Skittles. (“Because,” said a noticeably goofier and more relaxed Zuckerberg, “the future is delicious.”)
There were even more impressive effects, like the ability to point your camera at a scene of a cup of coffee, a plant, and a wine bottle. Tap on any of the objects onscreen and have Facebook’s platform recognize it for what it is — coffee or plant or vino — and offer up various effects or abilities, from creating a whiff of heart-shaped steam from the coffee cup to rendering an info card about the bottle of wine.
Zuckerberg was quick to caution that some of these products and effects were still in the works, and would take time (though a closed beta of the photo filters begins today). We may not have 3-D sharks swimming around our coffee tables for a while.
For one, a lot of the tech uses SLAM (or Simultaneous Localisation and Mapping), a technique that allows for a computer to render a 3-D map of a physical space. It’s possible to do this with a single lens, but it’s complicated, and requires more computational power. Dual-lens cameras make it much easier for a computer to get a grasp of the shape and depth of everything in front of it (think of the difference of judging how far away something is with one eye closed versus both eyes open). Dual-lens cameras are making their way to market, most notably the iPhone 7 Plus, but progress is slow — even Samsung’s latest flagship phone, the Samsung Galaxy S8, stuck to a single lens. (Google’s Tango is a project that attempts to solve much of the same problem, but has stumbled so far.)
So why so much about cameras and AR? Why essentially promise developers that the focus of your company is now building a much more technologically sophisticated version of Snapchat, and make it the cornerstone of your CEO’s big moment in the media?
As the New York Times writes, Zuckerberg feels that Facebook missed a major opportunity when it failed to build either a successful smartphone hardware (i.e., the massive failure that was the Facebook phone) or a smartphone OS, as Google did with Android.
It seems that Facebook saw the same thing happening with Snapchat, as the upstart company was one of the first to figure out that for many users, their smartphone is more a camera with messaging capabilities than anything else. And Zuckerberg and Facebook were determined not to miss this shift in how people use their phones.
So Facebook has spent most of the past six months slowly ripping off the core concepts of Snapchat. It brought its disappearing video stories to Instagram — and successfully got more people using it than they do Snapchat. It brought over instantly entertaining face masks and put them on Facebook Camera as Effects. It even copied Snapchat’s “geofilters,” where certain effects are only available if users are in a certain location. About the only thing Facebook hasn’t copied from Snapchat are Bitmoji, and that’s likely only because Snapchat bought the company for $100 million in mid-2016. But if Facebook can really deliver high-quality alternate-reality platform via your camera, it won’t just be taking Snapchat’s successful ideas — it will be innovating on top of them.
And it’s taking a much different tack than Snapchat. There’s a sense that Snapchat is Apple in the years before the App Store opened for business; it’s a closed ecosystem that invites certain partners it deems cool or stylish enough to partner with them, but keeps a tight lid on everything. Facebook is Microsoft; it’s building an open platform where it hopes to get as many developers as possible creating as many things as possible.
In its SEC filing, Snapchat described itself as a “camera company” — a term that seemed innovative at the time. Now — with Facebook’s 2 billion users, billions of dollars of cash on hand, and strong desire to become the leader in camera-based AR — that description may seem limiting. Facebook no longer seems to just want to copy Snapchat. It wants to beat it.
