Virtual Reality Opera — Saviour or Saboteur?
By Tom Nelson, Creative Producer, Royal Opera House
I have a confession to make: my 20-year-old self never liked opera. As a complete novice I found it completely impenetrable, even though I’d never seen one. I now work at one of the best opera houses in the world — the Royal Opera House in Covent Garden — and am completely won over by the power of this art form. When it’s done well, it grabs you by the jugular and never lets go.
Opera is the original immersive medium, where every dramatic tool is thrown into the mix. Our live stream and cinema broadcasts reach audiences across the globe, but Virtual Reality (VR) presents great opportunities (and challenges), with the potential of putting people right at the heart of our theatrical home.
Traditional filmmakers now strive to translate their skills of 2D storytelling to 3D, but these challenges have been natural to theatre makers for centuries. No wonder that places like the National Theatre are paving the way and championing VR’s storytelling capabilities — they have all the skills in house.
Here at the Royal Opera House, we have two great artistic Companies — The Royal Opera and The Royal Ballet — performing 29 different productions every year and a creative pool of hundreds who power this great machine.
I’ve long marvelled in particular at the professionalism of the Royal Opera Chorus — a singular body of remarkable individuals — who are able to blend their voices into one spine-tingling entity. Their sound hits you right between the ears as an audience member. How could we create an experience of being on stage with them, utilizing VR technology?
Our production of Verdi’s Nabucco provided an opportunity. The famous Act III ‘Va pensiero’ (Chorus of the Hebrew Slaves) features our chorus singing an anthem now as popular on Italian football stadia terraces as in the opera house. Director Daniele Abbado’s vision for this work sees the chorus centre stage and lit from above, a perfect staging to experiment with immersive filming.
We set ourselves two main technical learning outcomes with this project:
- Find a way of telling a compelling point-of-view (POV) narrative that puts the viewer alongside the chorus on our stage.
- Experiment with sound capture (more on that in a moment).
First things first, we went out and spoke to lots of VR production houses. The industry is fast moving, so we needed to get a steer on the latest thinking and tech. When we met with Jaunt VR, we knew they were on the same page as us. They believe in the power of narrative to take audiences on a journey through truly immersive content. On top of that, they’ve got some great hardware — specifically the Jaunt One camera.
The first ever professional stereoscopic VR rig, it resembles a sleek intergalactic Death Star, encircled by 24 cameras capturing in 8K resolution and supported by a robust post-production pipeline back at their base in Palo Alto, California. They were supported by a team here in London from Vision3, a company with considerable expertise in 3D storytelling.
The next challenge was creating truly immersive sound. Everyone’s trying to do it, but no one has really cracked 3D sound yet. Through a game engine like Unity, it’s much easier to control, but in real-world capture there are inherent challenges. A binaural head is currently the most common way of getting accurate positional audio.
The downside of this (and a major one at that) is that if we were to stick a binaural head on our stage and press record, it wouldn’t be a great listening experience that would match the world class musical standards The Royal Opera sets itself. In fact, it would be awful.
For one, the orchestra are more than 10 metres away, and recessed in the pit, so it can sound like you’ve got no accompaniment. Plus, with the distances involved, performers on stage actually have to sing ahead of the conductor’s beat so that the sound reaching the audience is beautifully blended. Our solution was to bring in a team from Soundintermedia working with our established Sound department on site to capture the performance, using float and radio mics — 59 channels in total. In post-production, we then created a powerful four-channel ambisonic mix (which has never been attempted before with live opera). I hope it strikes the balance between authenticity (that feeling of being there) and believability.
Logistically, the biggest challenge we faced was putting ourselves on top of an already insanely tight rehearsal schedule. Mess with it at your peril. Our production schedule required detailed liaison with all Technical departments. The set for Nabucco was also challenging — a raked stage covered in sand.
Our solution was to work with the fly tower team to suspend the camera from above in both rehearsal room and stage, allowing us to transition seamlessly from early rehearsals to performance.
Oh, and with all these challenges, I nearly forgot to mention the cherry on the cake. We only had one take to capture the final performance. No pressure then…
We’re just at beginning of discovering the possibilities of VR storytelling and the technology is still to break into the mainstream. Many industry insiders and the media are predicting Playstation VR to do this in the coming months.
Should an opera house do VR? We can’t say for sure yet, but we’d love to know what you think. See the full VR experience here. Or get the Jaunt VR App.
