Nearly three years ago, Facebook made a big bet on virtual reality with its purchase of Oculus, creating plenty of talk around the future of computing platforms. And on Tuesday when the social media giant kicked off its annual F8 developer conference, Facebook generated similar buzz around augmented reality (AR) when executives announced that apps billions of its users use daily are going to be getting new augmented reality capabilities. I already shared a few of my thoughts on this topic on Bloomberg TV, but wanted to dive a little deeper into Facebook’s virtual reality (VR) and AR latest developments and their significance.
Let’s start with AR.Ever since Facebook acquired MSQRD (reported to be between $70 million to $120 million) over a year ago for its AR masks, many have been waiting to see how they would leverage the company’s technology and open it up as part of its larger AR strategy. (Snapchat acquired Looksery a year earlier for a similar purpose.)
Now we know. With Facebook’s unveiling of the Camera Effects Platform, developers can build AR tools for the social media network’s in-app camera. At F8, CEO Mark Zuckerberg outlined three categories of AR:
· Information – Adding information atop the real world (e.g., directions over the street in front of you)
· Digital Objects – Adding digital objects onto the real world (e.g., a virtual post-it note on the fridge)
· Enhancement – Enhancing real objects with reviews, novelty filters, edits, etc.
Now that the developer community has a new technology to build on top of the world’s largest social network, here’s how I see them running with it:
Selfie masks and filters
Developers will create their own selfie masks and tap into Facebook’s data (e.g., a user’s location, the objects they’re filming) and depth-sensing technology. Some of these masks and filters will draw information from other apps too as we saw with examples Manchester United;s photo filter, which automatically draw in live match updates from other apps in real time. And as we saw with Nike, which could plug into Strava or Runkeeper to update your Nike AR selfie mask.
Games
These days, you can turn any flat surface into a game board to play an interactive AR game, which opens up plenty of possibilities for Facebook. The company is already using a state-of-the-art 3D mapping technique, SLAM (simultaneous localization and mapping), to recognize flat planes in the real world environment and superimpose digital objects on top of them. This leverages the inside-out tracking technology that is being developed at Oculus.
Graffiti
There will be strong appetite for drawing animated AR artwork on building walls or tables, which will allow people to add their personal tag to the real world around them. You could even leave virtual Post-It notes in relevant places (e.g., an AR sticky note on the fridge reminding your partner to pick up more milk).
Business AR
Businesses will be excited by the potential AR applications at their physical locations. How cool would it be to see social recommendations of your friends’ favorite dishes at a restaurant you’re visiting overlaid on top of the menu?
AI and AR
Developers building apps for Facebook can now tap into a user’s precise location, utilize FB’s object recognition and depth detection to create various AR novelty effects. Facebook is using convolutional neural networks to detect what’s going on in a photo or video, live in real time, making it possible to suggest contextual AR effects for that particular scene.
Being first can be important to developing a head start on developers wanting to build AR effects. And it’s clear by opening up their AR platform to outside developers, Facebook is going for scale. They don’t need to be responsible for developing a killer AR app, but they want it on their platform and will happily give away a ton of technology for free in order for that to happen.
As Facebook courts developers, it’s also making itself more attractive to brands. It can now offer them multi-faceted advertising and partnership opportunities: They can run their chatbots on Facebook Messenger, gain targeted insights through computer vision recognition of user-generated videos and develop intelligent brand-sponsored overlays on what users see through the camera. With one F8, Facebook added a whole other category that brands can make money off of.
Of course, everyone is wondering how this affects Snap, which just announced its own 3D filters. One area could be advertising costs. I see Snapchat as a closed ecosystem and brands currently pay upwards of $800,000 a day for targeted lenses and filters. Facebook will likely bring those costs down by making AR a more open ecosystem.
After these announcements, however, I don’t see Snap as a competitor from an AR perspective. I think Apple could be, however, if it can build these capabilities into its own developer platform on iOS. With the rumored depth-sensing, more advanced iPhone 8, it seems like a good possibility. But Facebook is THE social platform, and the social graph insights they can apply to AR experiences will be an advantage over iOS.
Amid all this excitement, it’s important to remember the bigger play is beyond smartphone cameras. Not coincidentally, Facebook revealed it is currently building AR glasses.
On the VR front, Facebook launched Spaces to make VR more social and interactive. Initially available in beta for Oculus Rift users with Touch controllers, Spaces is a cool hangouts-type app where you and several friends are in the same space and can watch immersive 360-degree videos at the same time. You can also have other friends’ avatars join and include live video calls of friends in the room too.
While this isn’t as flashy as the AR news, it suggests that a few interesting developments could on the horizon:
Platform-agnostic: It’s interesting that Facebook wants Spaces to be cross platform and not just tied to Oculus. This means you can talk to your Oculus, PSVR and HTC Vive friends in the same hangouts room. It certainly gives social VR companies like Altspace, PlutoVR, High Fidelity and VRChat some competition.
Holograms: I think Spaces has a long way to go, but realistic-looking holograms is the future here. Imagine being able to cohabit spaces with your friends, but in live hologram form, like Princess Leia from Star Wars. Companies like 8i (I’m an investor) are already working on holographic technology.
Samsung Gear: There weren’t really any innovations around 360-degree video, so it appears Facebook is de-emphasizing optimized VR for Samsung Gear, which is powered by Oculus.
With its purchase of Oculus, Facebook made it clear it intended to be the leader in VR. With its announcements this week, it’s clear it also intends to do the same in AR.
Sunny Dhillon is a partner at Signia Venture Partners, a venture capital firm based in San Francisco. Dhillon is an investor in 8i.
