Virtual? Augmented? Mixed? The Reality of why “xR” is the perfect catchall

Day to day, I help the talented team at BBC Education to build and improve the Information Architecture of products like BBC Food and BBC Bitesize. At every other moment, I talk about a personal passion: xR.

X Reality (XR or Cross Reality) consists of technology-mediated experiences that combine digital and biological realities. It encompasses a wide spectrum of hardware and software, including sensory interfaces, applications, and infrastructures, that enable content creation for virtual reality (VR), mixed reality (MR), augmented reality (AR), cinematic reality (CR), and more. With these tools, users generate new forms of reality by bringing digital objects into the physical world and bringing physical world objects into the digital world.

From Wikipedia aka ‘my favourite starting point’

TL;DR

Put simply, xR* is a catchall for VR, AR, MR and all the other expressions of reality we haven’t invented yet. Think of it as a spectrum. Here’s a picture to show you what I mean:

• When we’re in ‘Real Reality’ (RR), every object we perceive and action we take exists in the physical world.
• Virtual Reality (VR) refers to all encompassing experiences that mostly shut out the physical world.
• Augmented Reality (AR) exists somewhere in between, consisting of a mix of physical and digital elements within the same ‘space’.
• Mixed Reality (MR) changes depending on who you talk to. It’s another reason xR is useful.

That’s essentially it! But if you want to see how this diagram came into being, and the who influenced it, read on.

Defining an emerging spectrum

I’d heard of initiatives like OpenXR and CreativeXR, without realising what they meant. I didn’t start to grasp why the language was important until I heard it from Sophie Ashcroft at the VR Manchester Meetup.

Up until that point, I’d seen and heard people talking about Immersive Technology vaguely. They’d say things like:

“We’re hoping to do some AR or some VR… perhaps even Mixed Reality. But we’re not sure what yet”.

Part of my job is to seek clarity in classification and labelling, so this ambiguity had always irked me. Following Sophie’s spark, I went down a rabbit hole.

Making it visible

The concept of treating the virtual and the real as a single spectrum immediately made sense. It smashed all of the fogginess surrounding the medium and freed me to up to consider exactly what was going on in a particular immersive expression, without worrying about what particular category it belonged to. It felt like I had normalised my internal understanding and achieved a higher level of enlightenment.

Having my own understanding is all well and good; now I needed to try and communicate it. To enable wider understanding, we have to make ethereal concepts visible. Even if what we put out there is wrong, it gives us something to pull apart and rebuild more ‘correct’ than when we started.

So, I branched out from Wikipedia and looked at articles from the wider tech industry. It was through TechCrunch that I discovered the first hint of coalescence, with a simple sketch:

Credit: TechCrunch/S. Somasegar and Linda Lian

I quickly made my own, digital version and started using it in internal presentations and lightning talks, whenever I got the chance to talk about xR.

It’s basically identical, I know. But all great artists steal, right?

More recently, I wanted to see if anyone else was expanding on the xR spectrum concept to add more clarity. A short Google search away was a tweet from Kent Bye — a VR journalist I was already familiar with through his work for RoadToVR and the Voices of VR Podcast:

Unfortunately Google aren’t resurrecting the once-popular MTV Show

At Google IO, Clay Bavor (VP of Augmented & Virtual Reality at Google) presented a beautifully simple extension of the spectrum. He succinctly demonstrated how the proportion of computer generated objects intersects with the real world as you sweep across the spectrum. “Brilliant!” I thought. I could surely just reproduce this (with appropriate credit), label it “xR” instead of “immersive Computing”, and be done, right? Not quite.

Making it robust

I’m lucky to be part of a team of UX Architects that are always ready to bounce ideas around. So, I threw all of my thinking into our Slack channel to get some constructive criticism. Some of the points raised included:

1. TechCrunch and Google placed ‘Real Reality’ at different ends of the spectrum. Where should it sit?
2. Is ‘Real World’ a confusing term when you’re talking about a spectrum that includes so many concepts that include the word ‘Reality’?
3. How might someone that has accessibility needs serviced by digital technology perceive their baseline state on the continuum?

Through this, I decided:

1. ‘Real Reality’ should sit on the far left (at least in languages that read left-to-right) as this is the more ‘natural’ base state upon which everything is an built on top of.
2. ‘Real’ as a proportion of total reality perception is potentially confusing. Better to think in terms of ‘physical’ and ‘digital’.
3. It arguably muddies the water to include established digital affordances (screen readers, speech generating devices or indeed any form of digital content delivery) on the same continuum as those that mesh intrinsically with the immediate physical world. This is a spectrum that exclusively considers a tightly coupled combination of the digital and the physical in 3D space.

Now it felt like I was getting close enough to consolidate.

Making it real

All I needed, really, were a few exemplars to solidify the thinking and give people a real frame of reference. I have a HTC Vive at home, and one of my favourite purchases is ‘Richie’s Plank Experience’. Very simply, it places you in a virtual lift. The doors open to a view 50 storeys down to the bustling street below. In front of you is a plank, and the goal is to walk to the end of it.

Importantly, the experience contains a tool to conveniently map a real plank into the virtual world. This provides a ‘passive haptic’ feedback that enhances the sense of presence and makes walking to the end of the plank a viscerally difficult task. Your rational mind (“I am stood on a plank on the floor in my house”) battles your lizard brain (“wHy ArE yOU dOInG ThIS yOU’rE GoiNg tO dIE?!”) to an ambivalence it’s hard to communicate unless you try it.

To me, Richie’s Plank Experience is a perfect example of why the xR spectrum makes sense. Yes, it’s mostly virtual. But the addition of a real-world plank turns it into an ‘Augmented Virtuality’ that creates something so much more powerful than a fully Virtual experience could be on its own.

Towards the other end of the spectrum, BBC R&D recently released ‘Civilisations AR’, an experience that brings historical artifacts into your immediate physical world (via the intemediary of your smartphone). Once you start adding examples to the spectrum, the nuance of each experience is more easily expressed:

In the vicinty of Richie’s Plank Experience, experiences can employ ‘active haptics’ similar to the BBC’s “Home: A VR Spacewalk”, which originally toured UK museums with a dynamic vibrating chair. In the Spacewalk, virtual movements were carefully mapped to real-world vibrations, hijacking a user’s senses in a way that makes them almost (but not quite) believe what they’re doing is real.

Moving right on the spectrum from Civilisations AR, it’s easy to see the difference in an experience like RoboRaid on the Microsoft Hololens, which enables an experience with a greater variety of digital objects that more tightly interact with the user’s physical space.

An important note: the axes on the xR spectrum do not share any correlation to ‘Presence’ — that is, the degree to which a user feels any given experience is ‘real’. Indeed, the experiences that feel the most real are often those that take advantage of physical props mapped to digital counterparts.

This is may not be the answer, but it works — for now. I’m at least confident enough to share it with BBC UX&D next week and hope to make the case for ‘xR’ as the preferred term when referring to the immersive paradigm that is now establishing itself.

On a personal level, I believe xR has the potential to change the world in so many ways that you’d be forgiven for missing (especially as I imagine most people are significantly less obsessed than me!). We’re already taking people to places they’d never be able to go. We’re giving those that have lost something the chance to (at least temporarily) feel it again. We’re understanding (and maybe even diminishing) our unconscious bias. We’re providing layers of information that mean we accomplish tasks faster and more accurately. And (if Magic Leap lives up to the hype) merging our physical and digital worlds in a way that will help us break free from the little handheld rectangles of pixels we’ve come to habitually rely on.

Thanks again to everyone credited for helping me to make sense of a potentially confusing new(ish) world. And, if you have any thoughts or feedback on where I’ve got to, feel free to get in touch.

*On my travels, I also stumbled upon an alternate view that equates xR with “Extended Reality”, but I’m going to conveniently ignore that for now (at least until that Wikipedia article gets longer).

Thanks to Dan Ramsden, Michelle Collier, Oana Secara, Barry Briggs, Jonathan Halkett and Katie Roberts for their invaluable critique.