You are viewing content from a past/completed QCon -

Presentation: Evoking Magic Realism with Augmented Reality Technology

Track: AI Meets the Physical World

Location: Cyril Magnin II

Duration: 1:40pm - 2:20pm

Day of week: Wednesday

Slides: Download Slides


At Niantic one of our missions is building experiences that are shared and social. We’ve seen how playing together has made an enormous impact in engagement in our games. Our players tell us that besides having fun, they have found benefits in making friends and building communities. To do so, the AR interaction has to be natural to our senses. The digital world must obey similar rules as the physical world in order to create the suspense of disbelief in our brains. When this balance is achieved, players are immersed in this magical realism where they can have frictionless fun (check out Codename: Neon, one of our prototypes that was created to demo this). The technology just works as expected, obeying laws of physics. For example, players in Codename: Neon can harvest energy from the white pellets on the ground, and those are a shared resource–so if one player gets them, the other players can’t!
In this talk, we’ll explore how building a real world system is more a software engineering art. It requires making choices among a set of trade-offs. On one hand, we have complex computer vision and machine learning algorithms that burn many CPU cycles; and on the other hand, we have the conflicting goal of getting the system to be as lean as possible since AR runs on resource-limited hardware such as wearables or mobile devices.

Speaker: Diana Hu

Director of Engineering & AR Platform @NianticLabs

Diana Hu, is the Director of Engineering and Head of the Augmented Reality Platform at Niantic Inc., creator of AR games like Pokemon Go, Ingress, and Harry Potter Wizards Unite. Diana leads the engineering team, building core technology that enables developers to create shared AR experiences that seamlessly blend the real with the digital. Previously, she was the Cofounder and CTO of Escher Reality, a startup that was acquired by Niantic. In the past, she served as the first data scientist for cloud television at Intel and Verizon Labs and eventually built the data science team that brought research projects into products. She holds seven patents in computer vision and machine learning and has a BS in and MS in Electrical Computer Engineering with computer vision and machine learning focus from Carnegie Mellon University.

Find Diana Hu at

2019 Tracks