How mixed reality will shape us
Mixed reality devices like the HoloLens will make our lives and work easier in a few years. Be it through holograms projected directly into the world in front of our eyes or the ability to easily meet with others in virtual space — live and in 3D. This next generation of computers, which we wear as glasses on our noses, is being further developed by Marc Pollefeys in the Mixed Reality and Artificial Intelligence Laboratory in Zurich. In the interview, he tells us where it is already in use and how long it will be before it is suitable for everyday use.
OLIVER BOSSE SPOKE WITH MARC POLLEFEYS
At Microsoft, you are working on the development of the HoloLens — in short, a computer that is worn on the head and projects information directly in front of your eyes. This means that, unlike a mobile phone, you have your hands free. But the list of benefits of the HoloLens is much longer, right?
That’s right. For decades we’ve been on a journey to make computers more personal, and Mixed Reality is a logical extension of this path. We have gone from punch cards to character-based interfaces, to graphic interfaces, to touch, language, pen and gestures. Moving the computer into the three-dimensional world where people have always existed is the next step in making the computer truly personal.
The HoloLens makes it possible to place virtual objects in the real world. So I could place a hologram next to my laptop, which I can then always see there with the HoloLens. Can you explain what opportunities this opens up?
Mixed reality technology such as HoloLens enables users to interact with holograms in the same way as with other physical objects, which is more in line with our natural instincts for communication. We have developed the HoloLens 2 with the workplace in mind and especially firstline workers. So front-line workers who work using their hands — whether in a factory or the operating theatre. In our opinion, this group has been let down by technology for years, if not decades.
With HoloLens 2, we create solutions that enable these people to work more efficiently and productively. The workforce is supplemented and improved — and not replaced. We see enormous opportunities for the HoloLens to solve the impending shortage of skilled workers and to enable effective processes.
The HoloLens can make our work easier by projecting information and holograms directly onto a machine, making complex workflows easy to carry out. To what extent is this already being put into practice?
The HoloLens is already in use in Switzerland and enables people and organisations to innovate and digitalise work steps. For example, consider the world’s first spinal surgery navigated with our HoloLens at Balgrist University Hospital. Especially in the contactless era, the importance of remote servicing of industrial equipment or vehicles is increasing. The potential is enormous, too, especially as all possible areas of application are not yet foreseeable.
Will we soon all be able to repair our cars or carry out surgical procedures without proper training thanks to the HoloLens?
No. As already mentioned, the HoloLens is designed to enable trained professionals to do more — and not replace them. This isn’t going to change in the future.
The HoloLens is already quite small and practical for the amount of technology it contains. Your vision is, however, to make it optically even more suitable for everyday use. To integrate it into normal glasses. How far have got with that?
We are not ready yet to replace the PC with glasses in everyday life. We still need to work on many things, such as being able to wear the HoloLens all day long. I estimate it will take a few more years for this generation of devices to come onto the market for daily use.
The benefits in everyday working life have already been mentioned. What benefits can the HoloLens offer us in our private lives?
People’s interest in communicating with each other in virtual space has increased during the pandemic. We expect that we will remain in this hybrid world. Video calling works well, but Extended Reality is better suited for larger groups. Here I see the advantage of meetings in a virtual space enabling more comprehensive communication — and bringing people closer to us, for example, via holograms in the HoloLens. This allows you to locate your fellow people in the room and to better perceive their gestures and facial expressions. Applications such as Microsoft’s AltspaceVR will drive forward such digital meetings and events.
* * *
ABOUT MARC POLLEFEYS
Marc Pollefeys (born in 1971) is Partner Director of Science at Microsoft Switzerland and professor of computer science at ETH Zurich. As such, he has co-published over 550 publications, primarily in the field of computer vision. As a child, the native of Belgium was fascinated by mathematics and Lego. So it’s not surprising that he has helped to create one of the most complex modern “toys”: the augmented reality HoloLens glasses.
