I am honored to be the host of the 2nd Exploring Future Reality event today by NYC Media Lab at our Viacom Times Square headquarters. The panel I will join is “What is Reality” — together with VR/AR experts like Prof. Ken Perlin (NYU) and Prof. Steve Feiner (Columbia).
It is a very “zen” topic so I guess I need to wear a bit of a sci-fi hat to think about it. Here are some of my random thoughts, love to hear your feedback.
Reality is… 100% Real
Let’s start with the easiest but overlooked (?) one. With so many technologies that we all use these days, it’s easy to forget we live in the richest environment to date one can get : > Everything you see and feel wherever you are currently is “right here, right now, and tangible.” The hotel room I am typing this in is real, the people in the hotel are real, the lamps and desks also. I can sense all of them (see, hear, touch, smell and (for food/drinks) taste). It is truly amazing. In my opinion, the 100% real world sets the bar for any virtual/augmented/___ reality project.
Reality is 100% Virtual
Next, let’s think about the most current technologies in VR. Systems such as HTC Vive and Oculus are great because it takes you completely into a world that is imaginary. You can be Super Mario one second or climbing the Himalaya next. The goal of a VR experience is to make you believe that you are really there — by making the places, the people, the things that you encounter as believable as possible (just like in your 100% real world). As of today, we are mainly achieving that goal by toying with what you see and hear largely. The next hurdle is to overcome the other senses. If you see something and grab it but it’s just “air” then you know immediately you are not in the the real world.
The Future Reality: Somewhere in Between
The world we are just starting to venture into is somewhere in between. It’s very exciting and really gives me a headache sometimes thinking about it.
The somewhere in between means: the various places/people/things are all mixed across space/time/senses.
Across Space
The Oculus Connect demo Zuckerberg gave is a good example. There, he, his colleagues and family were all across different locations (Mark on stage, Wife at work, colleagues backstage, dog at home) and interacting and taking selfies at his home. These have perhaps the most immediate applications in all areas including social, gaming and work collaboration. When you can cut down the distance to gather with other people and things, you will be popular because you save them time. In the not too distant future, we can perhaps witness people across loooong distances gather. For example, an astronaut on Mars discussing a problem with another one on the Space Station as well as a bunch of scientists across the globe (that saves many years!).
Across Time
This starts to be a bit more complicated for me. Time: Past, Present and the Future. Perhaps let’s cut out future for now as I don’t know where we stand with building a time machine…
So that leaves us with bringing places/people/things across different time periods. Why do we want to do that?? I actually can think of many examples that might be appealing. For now let’s just focus on one case: memories. What if we can go back and dance together with Michael Johnson back in the 1980s? Heading back to that amusement park with your parents when you were ten years old? Gather multiple generations of your family including long-passed grand parents to mingle and laugh? (today, the grand parents might just be holograms… tomorrow, they can be AI-simulated figures that talk/behave close to when they were alive)
Across Space and Time
If we amplify everything, things can get complicated really fast… The place you are at maybe from the past. The people — a mix of past and present. The things: some real and right there with you, some aren’t. Managing the user expectations of these scenarios will be the key challenge… Sometimes you might want to believe everything is real (but they aren’t). Imagine you are in an AR Harry Potter experience at a Theme Park (paid $ and got some AR goggles) — if Harry and other characters/magic pop up, you would expect to feel that they are as real as possible. However, there are (extreme) cases where knowing someone/thing isn’t real can be helpful. If you are a surgeon with a few others (some here, some not) operating on a real patient, and one person has to tie/cut something RIGHT NOW to save his/her life — you probably don’t want to count on the virtual person (that looks so real) who just… can’t actually do that job (!).
Across Senses
That last example led me to want to think about how to “classify or tag” what’s real and what’s not?? The industry currently including Microsoft and Magic Leap is trying to make every virtual object looks as real as they possibly can. But to date, most of them are 100% pixel and you can’t grab them. Next, perhaps we can see “blank objects” that are 100% real that got augmented / projection mapped (with pixels and so the look is altered) so they can be held, touched, squeezed etc. And after that, when you are across space with another person (e.g., the astronaut), perhaps you both are working with 10 different objects: 5 from Earth right where you are and 5 from Mars where s/he is. But you both can manipulate them as if they are all there with you. That, is perhaps he most exciting reality twist that we can get to in the not-too-distant future…
I am done thinking about what reality is for now… and back to 100% reality temporarily. Later!
