First, what is LiDAR? It is an acronym for light detection and ranging. Basically (oversimplifying), it pulses light at objects in a room and measures the time it takes for the light pulses to bounce back. By doing so, it can understand the distance objects are from the iPad and build a more detailed understanding of a scene or more realistic picture of a room. In some ways, it is in the radar family. Cellular towers (and TV crime shows) use similar approaches to triangulate location. LiDAR isn’t new, but it isn’t common in consumer devices — more so with law enforcement. Apple’s iPad Pro, with updated depth frameworks in iPadOS, can measure the distance to objects up to five meters away both indoors and outdoors. There is no Superman vision here — it can’t “see” through objects. 🙂

Why does it matter in augmented reality (AR)? AR experiences will be more realistic and interactive for a couple of reasons.

First, the people occlusion is better. What is that? (Here’s a definition of occlude.) Basically, one of the digs on AR historically is that it doesn’t feel realistic, especially when depth is involved. Objects look like they float in rooms. I throw a football to my friend and it goes through him. Experiences can even be confused with awkward juxtapositions of real and virtual objects. By understanding the depth and shape of objects better in a room, AR experiences can better place virtual experiences, especially as people or virtual objects move. If I can’t see through a human in real life, I shouldn’t be able to see through a human in a mixed reality experience.

Second, LiDAR gives you a better topology of a room. Long story short, this enables more realistic 3D experiences. Games get better. Furniture buying or design apps get way better. Measuring distances or a human’s height gets more fun — and accurate.

I’m going to check out some of the demos as part of my research on updating consumer AR experiences, so stay tuned.

Our latest take on AR is here. We’ll update it in late April with more of a consumer focus to showcase just how good consumer AR is today.