Apple Patent Teases Augmented Reality Mixed With Social Networking

Apple is by no means first to propose augmented reality. But their simple, social vision of the technology could be every bit as disruptive as touch screens.

Apple Patent Teases Augmented Reality Mixed With Social Networking

We all feel like augmented reality could power the next killer app, but how would it work?


A new Apple patent application hints at how Cupertino may be piecing together an answer. Their “synchronized, interactive augmented reality displays for multifunction devices” describes a technology that allows users to share and annotate live scenes simply by tapping on their touch screens.

The patent describes more than some Snapchatted photo with a text overlay. Its backend is actually translating 2-D images into 3-D scenes, meaning your annotations (which can include URLs) float in real space, and they can be viewed from a variety of perspectives as another person tilts and pans around the same scene.

The difference between Apple’s vision and most, however, is that it’s seemingly focused on real time, and it’s inherently social (elaborating primarily on the ability for two people to view and comment on the same space). So let’s imagine, for a moment, that this isn’t another defensive patent to block Google but an offensive take on the future of augmented reality. Reading between the lines of a dull patent application, what could Apple do with the vision they’ve presented?

Richer MMS or FaceTime

These days, multimedia messaging means that we might include a photo with a text message. Maybe we FaceTime. With this AR system, you could feasibly share a very rich, contextual view of the world around you. A friend is running late to the bar, so you take a video snapshot and point out where you’re sitting. Maybe a bit bored, you decide to annotate all sorts of other meta content, like where a waiter dropped a drink, or where two drunks got in a shouting match about The Wonder Years. More impressive still, stack this idea into a live concert venue. Everyone’s phone can become another perspective, generating a dynamic, multicamera experience for someone stuck at home.


Disrupted Maps

Google has sent their Streetview cars across the globe. But a few hundred million iOS devices could create a crowdsourced snapshot that puts Google’s work to shame. Imagine opt-in global sharing of your snapshots combining to create a mega-annotated earth. Or consider the possibility of individual curation–someone wants to license their tours of New York, so you can download their walking tour of the city. Pointing out the best unlabeled storefronts would be easy fodder for such an app.

Better still, remember the boon podcasts had with iPods and iPhones, just because they were a low-fi means to share your perspective? Now amplify that quest for self-expression with real world context. Augmented reality can be bigger than even a walking tour; it can provide a whole new way of viewing the world, through someone else’s eyes.

Richer Media Atop The Real World


Apple’s application talks a lot about user annotation–in text and URLs. But let’s look at how Twitter and Facebook treat shared URLs today. They actually pull images from them in a seamless preview, saving users a click into a browser. Map that philosophy onto Apple’s proposal, and suddenly, you can place not just text but images and video in 3-D space across environments. Multimedia merges with the physical world, creating repercussions in communications and advertising that we can only begin to imagine.

No doubt, some of these ideas might sound a bit pie-in-the-sky, but assuming Apple has discovered simple interaction models (and technologies) behind sharing digital content within the analog space, it’s not hard to imagine the company being the first to crack the nut of addictive AR. Ironically enough, if Apple made one mistake in the last 10 years, it was undervaluing the importance of social experience. But after one look at this application, it seems like social could drive their next 10–assuming they actually develop the product.

Read more here.

[Hat tip: AppleInsider]

[Image: Manhattan skyline via Shutterstock]

About the author

Mark Wilson is a senior writer at Fast Company. He started, a simple way to give back every day.