Mixed Reality Immersive Theatre
Last night I went to see War of the Worlds, an Immersive Theatre version of Jeff Wayne’s 1970s musical version of HG Wells’ victorian science fiction novel.
I’ve talked before about Immersive Theatre, a type of theatre performance in which the audience steps into and even takes part in the story world. This normally means a real world play with real world actors, but War of the World is by dotdotdot a company that mixes physical immersive theatre with VR.
Mixed Reality Performance
The show immersed you in a fictional world that had both real and virtual elements. You started out in a bar that was themed around the steampunk aesthetic we would normally associate with War of the Worlds. When you enter the actual show you guided into recreation of a late Victorian London street with an actress who explains to you the setting. You then move through other rooms and environments, which are all have very well designed threats and which mostly have real actors who guide you through the story.
But these real world elements are interspersed with virtual elements that take various forms. The first is quite like a traditional cinema show, we were sat in seats and then watched a 3D animated introduction, though it involved projections that went beyond the traditional screen. The later sections were fully VR. In some elements we wore an HTC VIVE with a backpack containing a computer. That allowed us to walk freely around a large scale VR environment (which seemed very large but was probably contained in a relatively small room). At other times we hand the headsets in a relatively confined space, for example while sat in a boat, standing in a hot air balloon and a church confessional (which I will talk about later).
All together this created a (fairly) seamless world that had the intimacy and realism of real world immersive theatre but allowed a scale and fantasy of battling martians that is only possible in VR.
Transitions and On-Boarding
The mix of real and virtual is vital to the piece and a key part of that is the transition between the real and the virtual. There are actually two transitions, from the normal, non fictional world, to the (real, physical) fictional world and from that physical fictional world to the virtual world.
The first is quite hard, we have to be go from the street, where we are chatting with friends in a normal, day-to-day way, into a fiction world that is over a hundred years old. Not only that, we have to be “on boarded”, we need to understand how the production works and in particular, technical things like how to get a VR HMD on and off. All this was done in a way that feels like we are already in the fictional world. I don’t think this part worked perfectly. I felt that the script given to the actress who on-boarded us (understandably) struggled to integrate a 19th century conversation with explaining. This left us in a slightly in-between state of not been fully in the story and not being fully normal. This shows that designing on-barding is really important, but it doesn’t all have to be inside the story, I felt that it might have worked better if the VR was explained out of character, before we entered the story.
The transition into VR worked a lot better. As I explained before when I was discussing Marshmallow Laser Feast’s We Live in an Ocean of Air, the act of getting into and out of VR is also important, an the crew were well trained into getting us in and out of VR quickly and managed to integrate it into the story pretty well (the first time by making it feel we were being kitted out to be in the army).
Real/Virtual Overlap
The integration of real and virtual worked best when the real and virtual spaces were the same. One example is stepping into a physical boat, entering a VR environment where you are sitting in the same boat and sailing off into the sea.
My favourite part was where you entered a church confessional: a small wooden cubicle with curtains on two walls. When you entered VR, the curtains were drawn back in turn, firstly to see a priest talking to you directly. After that the second curtain is draw to show a scene in the church and then an alien world. This worked really well because the whole time you felt you were in the same physical space. I could even reach out and touch walls that were in both the real and physical worlds. Even when this didn’t work perfectly because the tracking wasn’t exactly lined up, the situation felt a lot more real because I knew there was a physical wall there. The physical object really added to the embodiment of the experience.
Characters
Characters were another important aspect of the performance. In the real part we were with a group of other people and were guided by actors who acted in character and were dressed in 19th century costumes. The virtual part also included both other participants and characters.
The representation of other participants was probably the part that worked least well. We were all in avatars of characters in victorian costume that looked appropriate. our heads and hands were tracked and used to animate the avatars, which is fine in theory, but can be a problem if there are issues with the tracking, which there were. The characters often moved and walk very strangely, with some of them looking like they were lying down or doing a limbo dance.
This weirdness really broke the feeling of plausibility. I personally would have tried to rely much less on the tracking, after all we don’t know what the other people were doing. I think that having a nicely animated character that just followed the person around would have worked much better.
What did work very well was the non-player characters. Dotdotdot had used what looked like volumetric capture to record actors’ performances, which resulted in very expressive main characters that were just realistic enough while being slightly stylised with the rendering.
Medium of the Future
With this and their previous performances dotdotdot are doing some very cutting edge work. The way they combine real and virtual is really exciting. It’s not perfect, there are things that still need to be improved, but the shows are getting better and they are really developing the medium. I’m looking forward to seeing a lot more shows that work this way in the next few years.
This is part of a blog I have started to support learners on our Virtual Reality MOOC, if you want to learn more about VR, that is a good place to start. If you want to go into more depth, you might be interested in our Masters in Virtual and Augmented Reality at Goldsmiths’ University of London.
