MinisterioVR: the case study
In June 2015 we decided to venture through a door. We did not know where it would take us, but that did not worry us too much.
In June 2015 we decided to cross the door of the Ministry of Time and to go straight to the area of Analysis and New Projects of Radio Televisión Española (RTVE) to propose what would be the first trip into virtual reality of a public TV channel in our country.
This was the beginning of an epic adventure through roads, gates and portals, where we found allies along the way. Onza Entertainment, the producer of the series, went all in for the idea and Pablo Lara, their Transmedia Producer, joined the team to do something impossible: sneak into the Ministry of Time and make the world’s most impressive episode in VR.
“This content truly achieved the potential of what VR is capable of!” Dylan Flinn, director of investments at Rothenberg Ventures.
We weren’t the only ones excited about the idea. The cast, the production team, the writers, the showrunner… they all were involved from the beginning. For them, this idea made sense: in the series, the characters can only travel to the past. Together, we were moving everything to a future where viewers are protagonists and everything revolves around them.
In this VR experience, named “Time in your hands”, we are allowed to interact with some of the main characters of the TV show and walk with them through the Ministry. The viewer becomes a civil servant candidate at the Ministry of Time, so the person will have to take an interactive test (based on the historical/cultural content of the show) to prove whether they are suitable for the job.
The shooting
“Elegant and sharp. A perfect prototype for the future of interactive 360 production.” Drew Arnold, CEO at Lit, director of music videos for Beyonce, Kanye West…
The shooting was two days long, and it was done in the same scenarios in which the show is recorded. All of the team (make up, production, decor and lighting) was integrated into our workflow to remain as faithful as possible to the original show.
The first decision was a technical one: should we shoot it in stereoscopy or mono video? Video with volume or flat? There was no doubt for us: virtual reality must be immersive and we needed realism. The best way to get that is to have an environment around you that has shape, volume, depth, so it almost feels like you can touch it.
To shoot this type of footage we used our own rig with two high quality cameras. We trusted Antonio García- Serrano (Zakato 360) for the job. He is one of the leading experts in stereoscopy in the world (no exaggeration!). This system, which achieves a sense of depth that fascinates us, makes the whole process much more artisanal.
A 360º recording makes sense when each angle has something interesting for the viewer. We kept that in mind and it added a degree of difficulty, because we had to synchronize the actions occurring at different angles. At the end, it’s not just a matter of when the action begins, but also raccord of sights and dialogues. As Nicolás Alcalá said, “in a virtual reality shooting, the director loses half of his possibilities: he can’t force the viewer to look at something in particular; he becomes a theater director, where the most important things are the staging, the sound, the lighting”.
“El Ministerio del Tiempo is visually stunning and engages the viewer with interactivity. The quality of the production and the performance of actors were top-notch.” Soya Seo, Business development at River Studios
This system can record up to about 90°. That forced us to divide the scenes in four quadrants from the script up: characters couldn’t walk from one quadrant to another, since that would make the stitching process way more difficult. However, we found a way around this and got a character to cross from one side to the other by interposing props that helped to make the transitions smoother. For example, in the second sequence, in the cloister, we stitched the scene using a column so the character could move from side to side without noticing that they were recorded in two takes.
What did we lose in exchange for getting what many people in the industry have qualified as the best 360º stereoscopic video they had seen in their lives? Movement, for example. One of the original ideas was to make a forward tracker moving sequence, which was absolutely impossible because of the current state of technology.
The body as well. Your body. Originally we wanted you to look down so you could see your own body. Objects were collected by “your” hands. In fact, we shot all the sequences twice, one without body and another one with one of us dressed in a ridiculous green chroma full-body tights to integrate a digital body in postproduction. At the end, it was not possible due to deadlines and budgets.
We took with us a VR headset for the actors and the crew, to show them what VR is. In the end, a funny scene between an actor and the creator of the series took place and we decided to includ it as an easter egg hidden within the experience.
Post-production time
“The Ministry of Time” is the best use of stereoscopy in VR I have seen so far. It feels right, well balanced and it really allows you to get immersed in the story.” Juan Ignacio Cabrera Stereographer for “Star Wars: The Force Awakens”
After five months of preproduction, fourteen script versions and two days of shooting, it was post-production time, in which we were doing several things at the same time. On the one hand we started stitching the image and the 3D renders of the digital scenarios. Then, José Jacas integrated the VFX, using the original assets of the series that USERt38 gave us.
One of the creative decisions we made was choosing a type of transition from scene to scene different to what we had seen until then. We decided that all the scenes would be connected by objects that different characters give you. So, when we finally decided not to include your own body, we recreated these objects in 3D to work as a transition.
On the other hand, we had already started the development of interactive tests (we like to say, in fact, that what we did was integrating augmented reality into virtual reality). And while we were creating the texts, the screen designs, etc, our VR engineer Nestor Viña was developing the application itself after a very long (and much debated) user interface design process. The last step was to integrate the video and the binaural sound within the application.
This project has had a complex postproduction process because of three factors. First of all, we had to reach the quality of a prestigious television show and ensure that it was as similar to it as possible. The second challenge was to get its look and VFX quality in a production recorded in 360º . And finally, a handicap of using stereoscopy: we had to work with two images simultaneously, one for each eye.
“That’s the moment when my work begins. I received the 360º images in pairs, one for each eye and also 3D renders for both eyes. My job is to join both together, removing the chroma, replacing the roofs, deleting items that should not be seen (as the spotlights) and matching the color and light to simulate the 3D environments of the actual set. All this while ensuring that anything done to the image of one eye is replicated exactly the same in the other eye, but offsetting the horizontal deviation of that eye to avoid problems of stereoscopy- when you see it in 3D, you can’t notice anything unusual there.” -José Jacas, VFX
For this project, Jacas worked with Nuke (developed by The Foundry), a powerful post-production tool for working with stereoscopic video, which makes easier the task of replicating all operations made to one eye on the other one.
Furthermore, alignment corrections had to be done by hand several times. Once the actual background was aligned with the 3D one, and when green screens were erased, Jacas had to blend the edges of each other, creating extensions of the floor and the ceiling. The greatest difficulty of this part was to match the stereoscopic depth, to avoid a sudden jump in distance between the set and the 3D background. All this work is a trial-and-error process until everything is perfect.
In some sequences, as in Angustias’ office, we had to generate a 3D roof. We also had to “paint” shades around the roof to give it the feeling of being real. As a last detail, we added digital lamps to replace shooting lights.
Shooting lasted two days. The post-production process… a little more: four months from December until March. This was a real learning experience for everyone on the team, as it was the first time we combined 360º shooting with stereoscopy.
Attention focus: don’t miss a thing!
“This was hands down the best VR experience at SXSW this year. Incredible direction and technical work by Nicolás Alcalá and the guys at Future Lighthouse.” Barry Pousman, producer/director of all UN VR videos including Clouds over Sidra and former head of vr on Discovery Channel.
The attention focus is an issue that worries the whole VR community. In MinisterioVR we use certain narrative techniques for guiding the spectator that have been working for us. This is a first-person experience, you are part of the story, so the characters are speaking directly to you. This allows, after the first thirty seconds, where the viewer adapts to the environment, to follow the plot without getting sidetracked.
And, how can we guide the spectator from an attention focus to another one? We can do it using a visual element, something that catches your sight and guides you to the second focus. Binaural audio has a key role to in this too. As we said, we have filled each vision angle with interesting elements for the viewer. When we needed people to look toward a particular direction, we used the audio to guide them in a natural way.
On the set, we recorded discrete sounds: each sound is recorded separately and placed with a special software in a 3D space. This software is able to simulate the binaural effect with different audios and make us believe that every sound comes from a specific position.
One of our priorities has been to do a simple and intuitive interface, “the reality is virtual but still reality”. We developed the experience and the app at the same time. While we were shooting and stitching the images, we were designing the user interaction and the app, which is developed in Unity. Our engineer has created a lot of original code to develop all the experience.
In fact, we have tried to disappear, we want to let the user enjoy the experience. We want them to live the Ministry from the inside, without the feeling that we were telling them what to do.
All this process has taken us 9 months and is driving us, as in the VR experience, to the door 2046: a door to the future which is being opened for us. Creating “Time in your hands” has been an awesome experience and we are really proud of our work. But it is not just that: the fan reaction has been overwhelming, as has been the reaction from people that had never watched the show, even people of the VR community has congratulated us.
The door 2046 has opened to let us enter into a future that will be bright. Shall you come with us?
—
Download the app:
IOS: http://apple.co/1XRQYmP
Android: http://bit.ly/20URbVT
Gear VR: http://ocul.us/25b6lJI
