What is augmented reality, anyway?

Augmented reality systems show virtual objects in the real world – like cat ears and whiskers on a Snapchat selfie, or how well a particular chair might fit in a room. The first big break for AR was the “Pokémon GO” game, released in 2016 with a feature that let players see virtual Pokémon standing in front of them, ready to be captured and played with. Now, technology companies like Microsoft^[1] and Mozilla^[2] – the company behind the Firefox browser – and even retail businesses like IKEA^[3] and Lego^[4] are exploring the potential of AR.

Where I do research^[5], an AR lab^[6] at the University of Michigan School of Information, it seems everyone knows about AR and is excited about the technology becoming popular among the general public. My colleagues and I watch videos of impressive AR demonstrations^[7], try out new applications^[8] and play with new devices^[9]. The research community’s enthusiasm may be why several experts – including some I talk with – say they expect AR to be commonplace in five years^[10], or envision AR glasses replacing smartphones^[11] within a decade.

But as an AR researcher with expertise in both industry and academia, I disagree with those optimistic views. Most people in the U.S. haven’t heard of AR – and most of those who have don’t really know what it is^[12]. And that’s just one barrier between augmented reality today and a future where it is everywhere. Overall, there are three major challenges to be overcome.

Hardware difficulties

When I first tried AR glasses three years ago, they quickly overheated and shut down – even when trying to do something fairly basic, like placing two virtual objects in a room. While there has been a lot of improvement in this respect, other problems have emerged. The HoloLens system – one of the most advanced AR headsets – essentially requires a user to carry a Microsoft Kinect system and a computer on their head, which is quite heavy^[13] and limits the user’s field of view^[14]. A different issue are AR experiences that work across systems^[15].

Even “Pokémon GO,” the most popular app that actually uses AR, drains smartphone batteries extremely rapidly^[17]. And the AR function doesn’t make the game much better – or really different at all – though it is neat at first to see a Pikachu standing on the lawn in front of you. With so little benefit and such a severe hit to device performance, every player I know, including me, has turned off the AR mode.

Lack of real uses so far

Just as people turn off AR in “Pokémon GO,” I’ve never seen or heard of anyone actually using IKEA’s furniture app^[18] as it’s allegedly intended; the app has just 3,100 reviews in Apple’s app store, far fewer than the 104,000 for “Pokémon GO.” It’s supposed to be useful to people seeking to redesign their living spaces, letting them use their smartphones to add virtual furniture to actual rooms.

Apple and Google have released AR toy and demo apps built with their new platforms ARKit^[20] and ARCore^[21] – such as playing with virtual dominos^[22]. They are engaging, and the 3D models look great. They do what they’re designed to do, but their functions aren’t especially useful.

This is partly due to the fact that AR, like the internet, is just a basic technology that needs people to create uses for it. The internet started as Arpanet in 1969^[23], but began to grow widely only when Tim Berners-Lee invented the “World Wide Web^[24]” – a now-dated term – in 1989^[25]. And it wasn’t until the 2000s that regular people who used the internet could also create online content^[26] for others to consume. That level of development and innovation has not yet happened for AR, though Mozilla is taking initial steps^[27] in this direction by trying to bring AR to everyday web browsers^[28] like Firefox.

Marketing challenges

Even people who use Snapchat don’t think of it as an augmented reality app – though that’s exactly what it is. It’s AR technology that figures out where to put the dog ears, heart eyes or whiskers on their friends’ faces – and sends rainbow vomit out of their mouths. People who don’t know what augmented reality is, or who have never consciously experienced it – even if they use it daily – aren’t going to make a purchase just because a product has some AR capability.

There’s also some confusion in labeling and marketing of AR technologies. Many people have started to hear about virtual reality, which is generally an immersive fully virtual world that doesn’t include aspects of the user’s real environment. The distinctions get fuzzier with mixed reality – sometimes labeled “MR” but other times “XR.” Originally the term meant anything in between a fully real and a fully virtual^[30] experience – which could include AR. But now Microsoft is saying products and apps are MR if they provide both augmented and fully virtual experiences^[31]. That leaves customers unclear what’s being advertised – though they’ll know it might not be very useful and may run their phone batteries down quickly.

I’m with my AR-optimist friends and colleagues in seeing a lot of potential for the future, but there’s a long way to go. They – and I – are already working hard on making the hardware better, finding useful applications and clarifying product labeling. But it will take lots of this hard work and probably many more years before mainstream America lives in a truly augmented reality.

References

1. ^{^} Microsoft (www.microsoft.com)
2. ^{^} Mozilla (research.mozilla.org)
3. ^{^} IKEA (www.theverge.com)
4. ^{^} Lego (www.forbes.com)
5. ^{^} do research (scholar.google.de)
6. ^{^} AR lab (www.mi2lab.com)
7. ^{^} videos of impressive AR demonstrations (www.youtube.com)
8. ^{^} try out new applications (vimeo.com)
9. ^{^} play with new devices (vimeo.com)
10. ^{^} commonplace in five years (arstechnica.com)
11. ^{^} glasses replacing smartphones (www.valuewalk.com)
12. ^{^} don’t really know what it is (www.valuewalk.com)
13. ^{^} quite heavy (thenextweb.com)
14. ^{^} limits the user’s field of view (mashable.com)
15. ^{^} AR experiences that work across systems (dl.acm.org)
16. ^{^} AP Photo/Elaine Thompson (www.apimages.com)
17. ^{^} drains smartphone batteries extremely rapidly (mobileenerlytics.com)
18. ^{^} IKEA’s furniture app (itunes.apple.com)
19. ^{^} IKEA (www.ikea.com)
20. ^{^} ARKit (developer.apple.com)
21. ^{^} ARCore (en.wikipedia.org)
22. ^{^} playing with virtual dominos (www.youtube.com)
23. ^{^} started as Arpanet in 1969 (en.wikipedia.org)
24. ^{^} World Wide Web (webfoundation.org)
25. ^{^} in 1989 (home.cern)
26. ^{^} create online content (www.oreilly.com)
27. ^{^} Mozilla is taking initial steps (blog.mozilla.org)
28. ^{^} bring AR to everyday web browsers (mixedreality.mozilla.org)
29. ^{^} dennizn/Shutterstock.com (www.shutterstock.com)
30. ^{^} anything in between a fully real and a fully virtual (medium.com)
31. ^{^} both augmented and fully virtual experiences (www.microsoft.com)