Mixed Reality Immersive Theatre

Marco Gillies
Jul 15, 2019 · 5 min read
Image for post
Image for post

Last night I went to see War of the Worlds, an Immersive Theatre version of Jeff Wayne’s 1970s musical version of HG Wells’ victorian science fiction novel.

I’ve talked before about Immersive Theatre, a type of theatre performance in which the audience steps into and even takes part in the story world. This normally means a real world play with real world actors, but War of the World is by dotdotdot a company that mixes physical immersive theatre with VR.

Mixed Reality Performance

The show immersed you in a fictional world that had both real and virtual elements. You started out in a bar that was themed around the steampunk aesthetic we would normally associate with War of the Worlds. When you enter the actual show you guided into recreation of a late Victorian London street with an actress who explains to you the setting. You then move through other rooms and environments, which are all have very well designed threats and which mostly have real actors who guide you through the story.

But these real world elements are interspersed with virtual elements that take various forms. The first is quite like a traditional cinema show, we were sat in seats and then watched a 3D animated introduction, though it involved projections that went beyond the traditional screen. The later sections were fully VR. In some elements we wore an HTC VIVE with a backpack containing a computer. That allowed us to walk freely around a large scale VR environment (which seemed very large but was probably contained in a relatively small room). At other times we hand the headsets in a relatively confined space, for example while sat in a boat, standing in a hot air balloon and a church confessional (which I will talk about later).

All together this created a (fairly) seamless world that had the intimacy and realism of real world immersive theatre but allowed a scale and fantasy of battling martians that is only possible in VR.

Transitions and On-Boarding

The mix of real and virtual is vital to the piece and a key part of that is the transition between the real and the virtual. There are actually two transitions, from the normal, non fictional world, to the (real, physical) fictional world and from that physical fictional world to the virtual world.

The first is quite hard, we have to be go from the street, where we are chatting with friends in a normal, day-to-day way, into a fiction world that is over a hundred years old. Not only that, we have to be “on boarded”, we need to understand how the production works and in particular, technical things like how to get a VR HMD on and off. All this was done in a way that feels like we are already in the fictional world. I don’t think this part worked perfectly. I felt that the script given to the actress who on-boarded us (understandably) struggled to integrate a 19th century conversation with explaining. This left us in a slightly in-between state of not been fully in the story and not being fully normal. This shows that designing on-barding is really important, but it doesn’t all have to be inside the story, I felt that it might have worked better if the VR was explained out of character, before we entered the story.

The transition into VR worked a lot better. As I explained before when I was discussing Marshmallow Laser Feast’s We Live in an Ocean of Air, the act of getting into and out of VR is also important, an the crew were well trained into getting us in and out of VR quickly and managed to integrate it into the story pretty well (the first time by making it feel we were being kitted out to be in the army).

Real/Virtual Overlap

The integration of real and virtual worked best when the real and virtual spaces were the same. One example is stepping into a physical boat, entering a VR environment where you are sitting in the same boat and sailing off into the sea.

My favourite part was where you entered a church confessional: a small wooden cubicle with curtains on two walls. When you entered VR, the curtains were drawn back in turn, firstly to see a priest talking to you directly. After that the second curtain is draw to show a scene in the church and then an alien world. This worked really well because the whole time you felt you were in the same physical space. I could even reach out and touch walls that were in both the real and physical worlds. Even when this didn’t work perfectly because the tracking wasn’t exactly lined up, the situation felt a lot more real because I knew there was a physical wall there. The physical object really added to the embodiment of the experience.

Characters

Characters were another important aspect of the performance. In the real part we were with a group of other people and were guided by actors who acted in character and were dressed in 19th century costumes. The virtual part also included both other participants and characters.

The representation of other participants was probably the part that worked least well. We were all in avatars of characters in victorian costume that looked appropriate. our heads and hands were tracked and used to animate the avatars, which is fine in theory, but can be a problem if there are issues with the tracking, which there were. The characters often moved and walk very strangely, with some of them looking like they were lying down or doing a limbo dance.

This weirdness really broke the feeling of plausibility. I personally would have tried to rely much less on the tracking, after all we don’t know what the other people were doing. I think that having a nicely animated character that just followed the person around would have worked much better.

What did work very well was the non-player characters. Dotdotdot had used what looked like volumetric capture to record actors’ performances, which resulted in very expressive main characters that were just realistic enough while being slightly stylised with the rendering.

Medium of the Future

With this and their previous performances dotdotdot are doing some very cutting edge work. The way they combine real and virtual is really exciting. It’s not perfect, there are things that still need to be improved, but the shows are getting better and they are really developing the medium. I’m looking forward to seeing a lot more shows that work this way in the next few years.

This is part of a blog I have started to support learners on our Virtual Reality MOOC, if you want to learn more about VR, that is a good place to start. If you want to go into more depth, you might be interested in our Masters in Virtual and Augmented Reality at Goldsmiths’ University of London.

Virtual Reality MOOC

A blog about virtual reality tied to our Coursera…

Marco Gillies

Written by

Virtual Reality and AI researcher and educator at Goldsmiths, University of London and co-developer of the VR and ML for ALL MOOCs on Coursera.

Virtual Reality MOOC

A blog about virtual reality tied to our Coursera Specialisation on VR

Marco Gillies

Written by

Virtual Reality and AI researcher and educator at Goldsmiths, University of London and co-developer of the VR and ML for ALL MOOCs on Coursera.

Virtual Reality MOOC

A blog about virtual reality tied to our Coursera Specialisation on VR

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store