EXTENDING REALITIES — BEYOND THE METAVERSE HYPE [2022 UPDATE]
PREFACE: This is an updated version of the article from March 4, 2017. I have consciously chosen to use the word Extended Realities rather than ‘The Metaverse’ as it implies the Plurality of experiences & platforms at the increasingly porous boundary between our physical & digital experience of the world. Besides that, Metaverse connotations are deeply tied with dystopian SciFi visions (esp via the writings of Neal Stephenson, William Gibson, Bruce Sterling etc) as well as more recent attempts by Facebook/Meta to appropriate the term.
Virtual, Augmented & Mixed/Extended Reality technologies will change so much of what we do today, & many things that we struggle to imagine yet. Fast-forward a decade of breakthroughs in neuroscience, artificial intelligence, robotics & the bleeding-edge development of VR, AR , XR — & all the new ways it is set to affect how we might processing & creating ‘knowledge.’ But, also, all of this innovation equally furthers the dangers of surveillance capitalism/totalitarianism, disinformation & cyber warfare.
It is becoming increasingly hard to disentangle how the techno utopian promises & the social/political/cultural dystopias they bring about. The main reason why ‘wondrous’ innovations tend to lead to oppressive applications is because of the exclusion of the marginalized peoples in the design process, the top down approach & the disregard of the most privileged towards the ethical concerns of the ones that tend to be the most harmed by such non-inclusive design & deployment. For broader context, I would recommend reading the brilliant works (& watching the talks) by Ruha Benjamin, Safiya Noble, Sascha Constanza-Chock, Ramesh Srinivasan, Kat Holmes, Sara Hendren, Zeynep Tufekci, Mary L. Gray, Sidharth Suri, Jillian C. York, Meredith Broussard, Peter Pomerantsev — who have all shaped my own perspectives as futures researcher & designer.
FUTURES BEYOND SCREENS
I’d like to invite you to reimagine what content could mean as Virtual, Augmented, Extended Realities become more & more real — the digital is increasingly becoming enmeshed with the physical world in such a way that our ‘reality’ becomes the digital vs physical ‘transparency’ mode that we choose.
The advent of digital realities is an opportunity for us to rethink the way we could be experiencing (dis)information. We are gradually leaving the glowing rectangular screens behind to step into computational space — ultimately, if the immersive media technology is to succeed on large scale, the future of immersive media technology is the future of computation.
As we go into this future beyond screens, we must pay attention to the context that our innovations shall be set in: we have been witnessing the loudest voices on the internet, in the media, & in politics screaming for alienation vs communication, enclosure vs openness, fear & intimidation vs curiosity & openness. In the light of resurgent authoritarianism & extremism, I cannot help but admit I have been somewhat troubled by how narrowly (& sometimes cynically!) so many scientists, technologists & media professionals have been approaching our responsibilities from the in terms of future ethics — our work is never ‘just’ entertainment or marketing or science or technology, we are, in fact, shaping the cultural/social/political/technological frameworks of the future — & the world that we do ultimately affects not just all humans but all living species on this planet.
More than ever before, we must think what ideologies the content we create could empower. More than ever before, we must think what behaviour the technologies we develop could enable. Stanislaw Lem wrote: ‘There are no answers, only choices.’ As much as we cannot predict the future (there is never a singular path to it & if someone says so you better know they are a charlatan) — what are the choices we are (un)willing to make that will shape the direction our collective future might take? What if instead of following the pre-existing status-quo future visions, we change & challenge some of them?
What made me fall for virtual / extended reality l the way back in 2013, is that it could give us that necessary distance to visualise & experience the complexities of physical reality in a new way. We have a precious opportunity to rethink how we could be interacting more profoundly with physical reality, not escaping it altogether.
TECHNOLOGICAL INNOVATION WITHOUT HUMANITARIAN EVOLUTION ALWAYS = OPPRESSION
Here are just some of the vital questions to ask: How much of the physical world do we want to change, modify or replace with the digital? What effect all this might have on our conscious perceptions & our subconscious bias & our very psyche & physical bodies? How much of our data and privacy are we willing to surrender to the providers of digital worlds & in exchange of what?
Unintended consequences of new technologies wreak havoc more severe & more widespread than we want to admit. We need to look no further than Charlie Brooker’s BLACK MIRROR or Keiichi Matsuda’s HYPER-REALITY to see the prototypes of the future where the merger of physical & digital space goes terribly wrong. These are important cautionary tales, & really digging in to understand what, why & how things could go wrong might be the one of the ways to try to do it right. But also: most Dystopia is very tempting as product roadmap for those who are willing to cash in as the world burns. & so the urgency for a collective push to do it more right than wrong is real — criticise how today’s way to approach future reality is broken –– yes –– but also to attempt to prototype actually inclusive & hopeful alternative ways.
The technological ‘progress’ might seem unstoppable, but none of it is inevitable or should be designed in one predetermined manner — sometimes it is indeed vital to question the very foundations of such ‘progress’. It might seem that we are at the beginning of a beginning of a technology that could be indistinguishable from magic, but it would only result in something actually helpful to humanity if we choose it to be so & resist the temptations of easy pathways of furthering the technological innovation approach predicated on surveillance capitalism & the ‘monetization of everything’.
Technological Dystopias & Technological Utopianism are NOT polar opposites but rather just reverse sides of the same coin — most Utopias are indeed built top-down on somebody else’s Dystopia.
XR AS THE ULTIMATE PLATFORM FOR MIND MANIPULATION
The media we create becomes us. The spaces we design inhabit us. The fictions we tell — if they are compelling at all — always bleed back into reality. Ideas have impact. How we speak matters. How we see matters. In fact, it all starts with how we see the world. Media is the modern day mythology — it lays the foundations of our civilisation & becomes the ideology upon which we build our societal values & inform our policies for generations to come.
In the early days there was quite a lot of hype around the notion of VR being ‘the ultimate empathy machine’. But I believe this perspective wilfully saw & presented only a fraction of the real scope of possibility of. No media technology has ever had as extreme a potential of both positively affecting, yes, but also harmfully manipulating our minds as immersive media technology does. If through the power of immersiveness we can make you empathetic, we can do something similar, & much easier, with intentions to foster intolerance & hate. Current day disinformation campaigns & even aggressive state sponsored disinformation warfare fades in comparison with what is to come once immersive media is the dominant media space. So, I’ll ask you all here, how can we remove these rosy glasses & being aware of all of these very real risks search for real strategies to counter them? What if we could stop seeing technology as this external thing to us, & start appreciating it as an extension of our own biology whose well-being is not just our responsibility but a welcome challenge?
IMAGINATION TO TRANSFORMATION
XR is the space that can manifest its positive potential only if we come together & work together toward a shared goal. The reason that almost a decade into the 3rd wave of immersive media technology there is still little truly extraordinary AR/XR/VR content is because we are all still working in disciplinary, cultural, social & (dis)ability boxes.’ Engineers with engineers, game people with game people, folks in the global north with other folks in the global north, & teams that overwhelmingly make very effort to include neurodivergent & disabled experts, sexual & cultural minorities etc.
Again, here I want to reiterate as strongly as I can: to create something truly new — experience, or the technology that powers it — we need to step out of our echo chambers & privilege the insights by previously most marginalized people & groups . We are held back not as much by early-stage tech, but by our fear of imagination, by our unwillingness to have a conversation outside our comfort zone, & our inability to really collaborate across the artificial boundaries used to divide us. We need to open our imagination, convert it to conversations, to collaborations, creating outcomes of transformation.
HUMANITY, AUGMENTED
We have created augmentations of ourselves for as long as we have existed. From slingshots to satellites, from eyeglasses to AR glasses, from crutches to exoskeletons suits to replacement of our biological tissues, etc., we are constantly evolving how we engage and augment ourselves. AI and digital assistants, & our future virtual dreaming, are all a part of the amazing, & sometimes terrifying, creative output of our species. It is up to us as a community, as it always has been, to continue to model the examples of doing good with what we create and focus on helping us all reach our potential.
To quote Audrey Tang & this poem of hers as a manifesto for a more compassionate future:
When we see ‘Internet Of Things’, let’s make it an internet of beings.
When we see ‘Virtual Reality’, let’s make it a shared reality.
When we see ‘Machine Learning, let’s make it collaborative learning.
When we see ‘User Experience’, let’s make it about human experience.
When we hear ‘The Singularity is near’, let us remember, the plurality is here.
We need more stories of plurality. We need more platforms for inclusion. We need more technologies for collaboration. In fact, could we embed collaboration in the very essence of our design?
SPACE TO CONNECT ACROSS BORDERS
I believe, not just we can, but indeed we must, shift the conversation from XR being a space of isolation to XR being a space for resistance, for connection, for belonging. Instead of escaping the physical reality, we can search for new pathways to co-presence and co-creation that transcend our physical reality and center inclusivity. Because we will have to find new ways to not just resist, but actually overcome the borders that are being imposed around us. As the angry men in ill-fitted suits want to build walls, we must build bridges.
I want us to envision VR/AR/XR as the most potent medium we have ever had to facilitate understanding, learning, connecting with each other across & beyond borders. I want to go as far as thinking we can indeed design the kind of XR that could actually make the physical borders between us increasingly obsolete.
CONTENT FRAME TO CONTENT MATRIX — XR IS A VISCERAL MEDIUM
Our short term goals have a real chance of killing our vision. Too often we reach for the low-hanging fruit even while being aware that could very possibly hurt the medium long term. Together, we can redesign future vision & maybe by working backwards we can be so much clearer about our present responsibilities, actions & choices. We cannot continue dragging the old media and its methodologies and our bad habits that go with it into this new media space we create.
Screens had to compete with all other distractions in our physical surroundings & we compensated for that distance with all the extra stuff that would terrify us in the real life: the often disturbingly fast pace, violent action, exaggerated visual effects. But what does it mean now that we won’t be watching it anymore, but actually being in it — how much more distressing might the effect of the fast pace, violent action & exaggerated visual effects might become? We have to shift our thinking from creating content within the frame of screens to creating the space we move through & interact with as our content matrix. We might need to profoundly reconsider the screen media rules & find ways to approach this new space with much greater compassion & tenderness expressed in visual & storyworld design, colors, shapes, & the overall user experience.
XR is an experience that requires trust. “You don’t even notice your subconscious mind — until you get scared, or horny. Inherited wounds are the background radiation of our lives.” When we experience digital space, just like with a physical space, we bring all of our subconscious trauma, that more often than not, we aren’t even aware of. XR viscerally taps into all that we are are, including our personal histories.
It is indeed easiest (& cheapest!) to achieve emotional reactions by triggering our fears. That’s why we’ve been seeing so many violence- & horror-based VR experiences in the early days of the medium. YouTube is full of the videos of people first-time-in-VR freaking out while being attacked by virtual zombies. Sure that seems funny, but only until that happens to you. The experience is virtual, but the fear is very real. Is the experience ‘just’ virtual if it causes real emotional, or even physical, harm?
We will witness the very real PTSD from the virtual/augmented experiences.We will see people mentally & physically traumatized by experiences that did not ‘really’ happen to their physical bodies at all. In the disproportionate amount of the VR/AR I have tried, cheap claustrophobic tricks have been employed to make me move through the virtual space. That Space, or action within it, or time closing in on me. If the only way you can motivate me is by threatening me, then this current version of the medium suffers from a serious case of arrested development. I invite you to support the media technologies that could actually feel safe & nurturing enough to help open us up, not continue closing us down.
XR DOES NOT EXIST WITHOUT INTERACTION
My creative partner Howard Goldkrand asks: How can we design our future as a compassionate collaborative network? All this might seem a daunting task today, I know; but if not us, then who, & if not now, then when? It’s much better to shape the medium in its early development stages than deal with harmful patterns as they become deeply entrenched.
With the advent of immersive media, we are about to fully realise the awesome & awesomely terrifying power of content. Content that we create could help us to move past the outdated, ossified structures of the existing society & visualise the more inspiring world OR could revive the most harmful prejudice & ignorance of the centuries prior. I want you to think how the new media technology infrastructures we design could enable inclusive, participatory story-world.
Throughout my life I have moved from creating still images to moving images to immersive spaces to expansive fictional worlds & what I have seen across all is not the discontinuity in the media, but a continuum of human experience: What you make me feel, more than what you make. It’s not as much about what you design, but what we can do in your design. It’s not as much about your subject matter, but how you approach it. Alysha Naples, who led the design of user experience & interaction at MAGIC LEAP says: Do not make something ABOUT something. Make something that IS something.
The more I work in the field, the more I realize that there isn’t really such a thing as a ‘grammar of extended reality.’ We are trying to establish format compatibilities & certain rules of engagement of interaction, but the bigger question here is: How do we craft ‘reality’ situations? How do we design possibility spaces? My personal number one rule for designing extended reality content/tools/platforms is a consciousness around engaging with our physical reality. Observing what makes us react to physical experiences, how we move through physical space, how we interact with physical objects — paying attention to these “laws of interaction” so we can bend & open them to an even wider array of possibilities in the digital/virtual space.
The most important thing to remember is that audience is not just audience anymore, & more even than a participator — the ones that are experiencing the virtual world ultimately are the creators of their own experience.
F*CK TECHNOFETISHISM
For the future of this technology to be as good as we dream it to be, the technology has to become transparent. Bruce Sterling says:
We can no longer allow ourselves to be hypnotized by the sense of technical novelty. We should look at it like it is already ‘passe,’ and create it from that point of view. It must be good without us considering that is is new. If an experience is only good because it is new, it won’t be good for long.
We do not have ‘a technology problem’. Accuracy is not the only measure of quality. In volumetric XR space, most seem to be pursuing the pixel perfect re-creation of reality. That is something we might sometimes need, but it is not the only thing we need, & certainly not what we need most. The simulation of physical reality will never be as good as the real thing — it is much more fruitful to focus our attention in imagining & designing experiences that augment the real world or allow to experience aspects of it inaccessible with a ‘naked eye’ — the work by Marshmallow Laser Feast illustrates wonderfully such approach. The MLF collective’s immersive experiences explore ecosystem complexity & interconnectedness & allow us to see the ‘invisible layer’ of it in such a way that gives us back a true sense of wonder & a real desire to understand & engage more with the living world around us.
It is vital that more time, energy & resources go to the R&D of content. Because XR is not hardware. XR is not platforms. XR is an experience. XR is a digital space that you step out of with physical memories. VR is a journey, leading ultimately, into experiencing your own inner space.
INDIGENOUS ROOTS OF XR AS A HUMAN EXPERIENCE
XR is not new. It is simply a technology that can finally begin to echo the altered, augmented, expanded states that the Indigenous spiritual practices since time immemorial have brought humans into through their sacred rituals. Rituals that, since their inception, have had a goal of teaching us how to dive into the parts of our psyche unknown to our conscious minds so we could learn how to come closer to the world that we are inextricably linked with. I know this might sound somewhat esoteric, but that’s what ‘being in the space’ while knowing that that space is not there very much is — it is literally an out-of-body experience.
More than ever before, creation — be it artistic, scientific, or technological — needs to have some of that deeper, & also more humble & more informed approach towards our subconscious. I dream of social technology tools that could help us heal our wounds — as individuals, as cultures, as societies, as a species among species living in tangled web of life on this planet. The greatest innovations open new avenues for our perception of the world & our role within it — & heighten our sense of responsibility & reciprocity towards it all.
Our technology should not just be about solving ‘practical problems’ to increase material productivity, consequences on us & on the planet be damned. Creativity is not about devising smarter ways of selling more things people do not need. Creativity is about creating moments of meaningful experience. Experience that expands the human potential — creatively, emotionally, intellectually, physically. What we need is not more technology-driven experiences centering the wants of the few, but the experience-driven technology that inclusively serves the needs of the many. We need technology that can take us further, beyond our physical bodies, not to escape or replace them — but so that we can bring back that digital possibility-space into our physical lives.
NOW IS THE FUTURE
Everyday, with everything that we do, we create the culture of the future that we, & everyone else will be inhabiting. Our actions matter, but so does our inaction. If I can leave you with one question, it is: What is the future reality that you want to participate in creating?
Image #1: Eliud Lopez & Monika Bielskyte collaboration for @ProtopiaFutures. Image #2: Chris Bjerre & Monika Bielskyte collaboration feat graphics by Kazuhiro Aihara for @ProtopiaFutures.
Further reading: Protopia Futures Design Framework
