Where ‘Black Panther’ meets ‘Black Mirror’: Our immersive future
How the next computing platform could change our lives
Princess Leia was my first hologram. As a little girl I was mesmerized by the 3-D projection of my Star Wars heroine and wondered, “How is that possible?” And then there was CNN’s hologram of Jessica Yellin reporting on election night in 2008 and Tupac’s holographic performance at Coachella in 2012. At the time, they struck me as amusing stunts, rather than the future of media.
The hologram that changed my perception of what’s possible (and what’s to come) appeared in the 2018 movie Black Panther, when Shuri, Wakanda’s chief technologist, conjured a hologram of Okoye from the palm of her hand and had a conversation with her. She was able to interact in real-time with a photorealistic 3-D digital representation of a person. It was like a video call without the phone.
Something clicked. Not only did I know it was possible, but it seemed inevitable. And I finally had a new pop culture reference to explain what I do.
An Immersive Journalism Journey
Before coming to the JSK Journalism Fellowships at Stanford University, I was part of a team of immersive journalists at The New York Times that produced stories in virtual reality and augmented reality. We experimented with interactive storytelling tools like 360 video, photogrammetry, volumetric video and 3-D data visualization. In an Emmy-award winning report, we created a 3-D model of the site of a chemical bombing in Syria.
As we were exploring these emerging technologies, the Times was also reporting on the dark side of tech: how social networks are being used to incite genocide and terrorism, how algorithms are spreading disinformation and hate, and how our personal data is being collected and sold for commercial purposes and political manipulation.
I started to think about our new tools in a different way. I wondered what unexpected horrors they might bring in the future. Would they turn our lives into a Black Mirror episode?
Exploring the Unknown Unknowns
Over the past five years, my work in immersive journalism has been guided by my expectation that some day we will interact with the news, not through the screens of our phones, laptops and televisions, but through 3-D digital information embedded in the real world. It’s called spatial media, mixed reality or immersive computing.
Clay Bavor, VP of Virtual and Augmented Reality at Google, says, “With immersive computing, instead of staring at screens or constantly checking our phones, we’ll hold our heads up to the real and virtual worlds around us. We’ll be able to move things directly using our hands, or simply look at them to take action… It’s the inevitable next step in the arc of computing interfaces.”
The VR and AR headsets that we have today are the first generation of immersive hardware. They are evolving quickly and will eventually (hopefully) look more like a hipster’s Warby Parkers than a cyborg’s swimming goggles.
These wearable computers will be able to see, hear, create, think and learn.
What could possibly go wrong?
That’s a sincere question. I believe that journalists have an important role to play in helping the public understand how these emerging technologies might impact our societal, political and economic structures.
I keep asking myself: In 10 years, what will we wish we had known now?
As Google, Facebook, Apple and Microsoft are building an immersive ecosystem, this is the time to carefully consider the potential risks and unintended consequences of our immersive computing future.
For example, when immersive headsets scan rooms inside people’s homes, who owns that data? What about biological data such as eye tracking and heart rate? What is collectible? How is it secured? Can it be shared?
The list of questions extends well beyond data and privacy. Will we know the difference between the real and the virtual? How could these platforms be used for disinformation? How will digital advertising be deployed when the real world is its canvas? What interactions should require consent? How could immersive wearables affect the central nervous system? How will biases inherent in AI systems such as natural language processing and computer vision undermine immersive computing?
What are the unknown unknowns?
During my year at the JSK Fellowships, I am working with others in journalism, engineering, design, public policy and social science to probe these questions and more. If you are a maker, a builder, a tinkerer, an ethicist, a futurist, a skeptic, an evangelist, or a thought leader of immersive computing, I want to talk to you. Please reach out to me at hopkinsm@stanford.edu.
