Since Mobile World Congress, where the reality on the show floor was often either virtual or augmented, I’ve been thinking quite a bit about the practical uses of AR and VR – particularly in government and a smart city context. It’s not just all fun and games, is it?

The example of changing a roller coaster experience with new settings delivered via VR glasses is really cool. Yes, you can imagine repeating the ride to experience catapulting through medieval battle, flying through a tropical jungle, or bobsledding down alpine slopes. But the practical side of us – or at least me – wants to know what else there is. And, fortunately, I have a colleague who has already been thinking of these things.

A few months ago, I had the pleasure of collaborating with JP Gownder on a presentation for Forrester clients in Geneva.  I presented on the ways to derive value from data and opportunities to leverage new insights service providers – clearly something top of mind for many of our clients. But alas JP’s presentation was much cooler, providing examples of how to derive real value from new technologies including AR and VR.  Since then I’ve being thinking about how the two are related.  And, in fact, they are.

Think about data and the insights derived from them. How do you visualize those insights?  Dashboards and reports filled with charts and tables are the traditional business intelligence tools, delivering historical performance and predicted trends to leaders often on a standard cadence. But what if the data and derived insights were delivered in real-time in the field – by augmenting reality or virtual reality (that is, augmenting or adapting virtual reality)?

A few examples illustrate the possibilities:

When a truck approaches a border crossing, surveillance cameras capture and process images from all angles. Meanwhile a customs and border protection agent addresses the driver. Wearing augmented reality sunglasses, the agent is able to see real-time translations of what the driver is saying (based on natural language processing and translation).  As the agent steps back to examine the vehicle, she gets another message – the results of analysis of the surveillance video taken of the truck: "Vehicle chassis is two inches lower than standard make. Search all compartments." 

A first responder in a helicopter searching a city for survivors after a hurricane can get information about damaged buildings and infrastructure as well as the progress of rescue efforts on the ground.  Wearing AR glasses, he is able to see details of the buildings, what types of buildings they are, possible occupancy levels, and labels that indicate which buildings have already been searched and cleared. As the helicopter hovers over a school, the pilot is able to see images of the school before the hurricane superimposed over the damaged structures, with estimates of areas of greater risk. Observation enhanced by data and predictions enable better direction to teams on the ground. 

In another scenario, a public works engineer uses data delivered through augmented eyewear or a tablet to identify faulty infrastructure, determine the cause of the problem, and identify a fix. Or a building inspector accesses the file of the building he is currently inspecting and receives information about the builder which might include previous code violations tipping him off to pay close attention in certain areas.

In virtual reality the insights wouldn’t be an overlay.  Virtual reality is more immersive typically replicating a complete environment. But we can also imagine the environment changing based on input or immediate feedback.  Field training for soldiers or flight training for pilots delivered via virtual reality can be adapted to focus on specific scenarios that are problematic for the trainee, narrowing in on the specific context or condition that might be troubling.  

So, in fact, our presentations – mine on deriving insights from data and JP’s on delivering new experiences through cool new technologies – were inherently complementary.  Nevertheless, his was still much cooler.

We hope to co-author a report on uses of augmented and virtual reality in government – and how these tools deliver insights to improve outcomes and better serve citizens.