Holograms and the the future of meetings

You don’t have to look very far to find inspiration about what would future meetings look like. Decades of Sci Fi, whether in cinematographic or written from, have explored the question. It’s interesting to see that it used to be the case for technologies that are nowadays omnipresent in the meeting room. Videoconferencing was introduced in movies as soon as the 50s before being a reality in the 90s, the most famous sequence probably being the videophone sequence in “2001 : a space odissey”, that become an iconic view of the future.

A more recent trend explored in many pieces of SF is the concept of holographic communication, communication with a full-scale person as if you were face-to-face. The most obvious reference here is probably Star wars with the blue transparent hologram introduced in “A New Hope” and consistently used in the rest of the episodes.

Apparently there’s a technical trade-off between transmitting 3D data from a different galaxy and having a full color palette

Many other works have used what has become a SciFi cliché, especially when they come with glitches and visual static. It has become a very visual and easy way to display futuristic-looking conversations in the future.

As often, SciFi drives technical progress. These holographic meetings could soon be a reality : check what we’ve being doing at Mimesys.

What kind of sorcery is this? Well, a combination of three different technological advances allowed us to build the software behind this video :

• volumetric capture : the “3D” aspect is possible thanks to the data we get from a depth camera (here the Microsoft Kinect v2). Depth sensors are promised to a fantastic future because they will be soon the center of a big competition race in the mobile world. Prepare for such a sensor in your next iPhone.

Yep, this is exactly what we’re building : a 3D motion capture format

• Live streaming : the internet has turned into a great infrastructure for streaming high resolution 2D content. We use that, yeah.
• AR and VR headsets : The most important brick ; don’t expect hologram technology in the next years (though we’re already compatible with one of the most promising ones). Our holograms aren’t holograms, there are 3D meshes projected through AR (Augmented Reality) or VR (Virtual Relaity) devices. Incorporating real people in virtual environments (VR) or in real ones (AR) becomes then much more feasible than having to deal with physical constraints. On the video, Davy is looking at me through a VR headset, the Vive. The VR headsets such as the Vive Pre or the Oculus Rift are already at a very high performance stage, whereas we’re not sure yet of the performances of AR headsets such as Magic Leap and Hololens.

Our promise is the following : tomorrow you’ll have holographic communications while being recorded by a depth camera and while wearing a VR headset/AR glasses or lenses. That is what we’re exploring with Mimesys. We’re closer than ever to perform remote face-to-face meetings.

VR remote meetings

The problem with VR meetings is obvious : you have a headset on, so you’ll look like this to your colleagues. Sexy.

But for a lot of usecases, this will be good enough! There are still thousands of applications that value spatial positioning, body langage more than facial emotions (think of training applications for instance).

AR Remote meetings

Where remote meetings will thrive is with AR. With AR lenses, you’ll be able to see your partners face and integrate their bodies in your current world. This has been more than well described in the book Rainbows End, where every meeting is an AR one.

Prototyping the future

In parallel to our development, we’re exploring and prototyping what could Augmented Reality meetings look like, with an advanced platform with 68 cameras. We’ll share the results very soon.

There are indeed still a lot of stuff to explore around this concept that Hollywood hasn’t, especially :

• how to adapt the space and furniture to get a perfect experience? Holographic communication is complicated to achieve. As in the early days of videoconferencing, you’ll need a fixed setup and perfect conditions. The limited FOV on the current AR headsets will also imply some additional constraints : don’t be too close to the person you’re talking with. This setup could be an interesting challenge for furniture companies : how do you design a chair with discrete markers for careful hologram positioning? how do you design a chair for virtual people only? Could you reconfigure a room to host different meetings at the same time?
• What’s the User Experience for holographic communication? The Star Wars experience is just the beginning. When you bring the virtual in the real world, there’s much more you can do in terms of inputs and commands. How would you share a virtual document with a holographic colleague? With real sheets of paper with markers on? Is it polite to change the environment during the meeting like in Heavy Rain? Is it polite to turn into an avatar during a conversation?

There are still a few years before requesting Obi Wan’s Kenobi’s help in 3D, but count on us to be among the first to try.