VR vs Video Conferencing
I’m writing this a couple of weeks in to the UK’s lockdown for the COVID-19 pandemic. This is my first chance to take a step back to think and write after two very hectic weeks of adjusting to teaching and research online (and, to be honest a very hectic 6 months, for lots of, mostly good, reasons).
It is also a chance to reflect on a couple of weeks of having almost constant meetings, but having them all remote and online. Like most people I have done the majority of these via a bewildering number of video conferencing systems (Skype- standard and business, Zoom, Microsoft Teams, Big Blue Button, Whereby, Discord, WhatsApp, WeChat, and probably others, it sometimes seems like every meeting has to introduce a new platform). Having VR headsets at home (plus friends and colleagues who also have them), I have also been lucky enough to have meetings in a similarly bewildering set of Social VR platforms (Mozilla Hubs, AltSpace, Rec Room, VR Chat, Big Screen and others).
I have spent many years thinking about the benefits and problems of Social Interaction in VR and how it differs from video, but this is my first chance to really experience what this means for an extended period in my real working life (as opposed to experiments). So here are my first reflections.
Space and Body Language
One of the biggest differences between video conferencing and social VR is 3D space.
How we use space is a really important part of social interaction. In a romantic dinner we sit opposite each other to give our partner our fullest attention. When working together, on the other hand, we might sit side by side to focus on the work itself. In a larger social group we might sit around a table, as equals, echoing King Arthur’s round table, but we might also form smaller sub-conversations by turning towards one person and not another. When teaching or giving a presentation, the use of space shows who should talk and who should listen: the person at the front is in charge, listen to them.
Space is the foundation our body language. We turn our head towards some one to make eye contact; we turn our bodies towards or away from other people to indicate who we are talking to; we gesture in a certain direction.
Video conferencing doesn’t capture any of this. Instead of sitting in a conference hall or around a table, we are reduced to a grid of images.
This image of a meeting of the UK cabinet is nothing like the social interactions we are used to in the real world. They aren’t around the conference table they would normally be having their meetings at. The prime minister can’t use eye contact to suggest who speaks next or give a subtle meaning, or any other of the myriad other cues that we effortlessly use to communicate.
Romantic Dinner for 20
The renowned VR researcher Jeremy Bailenson has written an article about why video meetings seem to be particularly exhausting, relative to normal face-to-face meetings (paywall).
His theory, based on a number experiments his lab has run, is that it is because we are looking at a wall of faces staring directly at us. This is a very intense social experience. In terms of the real world it is a bit like a cross between speaking in front of an audience and having a romantic dinner. The number of people can be as many as you would speak to in a formal presentation, but in the real world their faces are unlikely to be as close or as directly facing you. The only place where we have such intense face to face attention is a romantic dinner for two, an intimate situation you aren’t likely to have with colleagues or strangers, and which, by definition, you never have with more than one person. It is certainly nothing like a normal business meeting, where we sit around a table, only making eye contact with one person at a time, and even that rarely.
VR is a spatial medium
The big difference with social VR is that you are all in 3D space, which can make things much more like real world interaction. One of the main things I noticed about VR is how we could easily use space in social ways. We could turn towards the person speaking; a presenter could stand at the front of the room; we could form small group conversations by standing is a circle, just as we would at a party. To me this made large meetings feel more manageable, and smaller interactions were more social and fun.
This isn’t simply because we are in a 3D virtual world. Using a 3D virtual world on a 2D laptop screen is not the same. You feel like you are looking in on a meeting, watching it on TV. Instead, VR can make it feel like you are in the space.
Other aspects of body language are, of course, more limited. We can’t see facial expressions, eye contact or, in most cases, legs. What we do have is hands and head movements. Hands gesture are one of the most obvious components of body language, but I actually felt that the head was more important. Because the HMD tracks our head, our avatars’ heads move as ours do. Not only does this create a strong sense of “life”, but the subtle cues of turning to some one, or away are really important socially.
One good example, is when I was attending a big meeting where some one was giving a presentation, and I noticed that my colleague Sylvia was there. We could say hello by looking at each other and waving, just like in real life, all without disturbing the rest of the meeting. I could then go over and stand next to her, also as I would in real life, to show some togetherness. None of that would be possible in a video conference.
Let’s not get carried away
That’s not to say that current social VR is a perfect form of interaction.
Video conferencing can convey facial expression, which VR can’t at the moment. Though I would say that body language in video is often not quite right, for example, we can’t make eye contact properly If we look into another person’s eyes we are actually looking away from the web cam, so it looks like we are looking away from them. I believe body language and expression in VR will improve with new headsets that include eye tracking, mouth tracking and muscle sensing.
Also, though I followed Jeremy Bailenson in saying that video conferencing is exhausting, I wouldn’t claim that current VR isn’t exhausting. The reasons are different, but eye strain and mild nausea, among other issues, can make long bouts in VR a tiring experience.
VR is power
Another problem is access, you cannot join a VR meeting if you don’t have a headset. Most social VR platforms address this by providing a desktop version, but this can be problematic. As I mentioned above, the experience is very different, looking in on a meeting rather than being there, but there are also deeper worries. Many years ago now, Anthony Steed, Mel Slater and team at University College London did a study of group collaborative tasks in which some participants used an immersive HMD and others joined the environment via a desktop interface. They found that the way people viewed the world affected the group dynamics. The person in the HMD almost always took a leadership role, dominating the task. This leads to the disquieting possibility that your social influence in VR worlds could depend on how much you can pay for a headset, particularly as new headsets add new body language features.
Social Interaction of the Future
During the COVID-19 crisis we are all becoming much more aware of the technologies for remote social interaction. While we know that at the end of the crisis we will be able to see people face-to-face again, it is also clear that our future will involve more mediated interaction than the past. No technology is perfect, though VR has the potential to improve a lot and overcome some of the limitations of video conferencing. It is still very important to understand the affordances of these different media, and what they are good for. There has been decades of research on this in the fields of VR, media psychology and Computer Supported Collaborative Work (CSCW), this looks like a very good time to look back at them.
This is part of a blog I have started to support learners on our Virtual Reality MOOC, if you want to learn more about VR, that is a good place to start. If you want to go into more depth, you might be interested in our Masters in Virtual and Augmented Reality at Goldsmiths’ University of London.
