A 360° 4D VR Experience. Stories from the set.
In June last year, Seat asked me to help them create a 4D Virtual Reality experience about the powerful cultural bond between the Seat brand and the city Barcelona. It was an amazing collaboration, but also one of the most challenging 360° shoots I have ever directed.
I worked with Ogilvy Spain, Wildbytes and Sergi Capellas to make this idea come to life.
Here are some stories from the set.
Traditional film crews are highly organized, and specialized in their field. There’s very little overlap in responsibilities, everyone knows what they need to do, and their strength is in repetition. On the other hand, we have the innovation teams that build new 360 camera rigs, and who improve and embrace chaos. Their skills are exactly the opposite.
Live Action vs. 3D
At the core of the challenge was the ambition for the camera to be ‘flying’ through the city of Barcelona, showing the most iconic locations, in a constant motion, as the viewer chases a real life Seat Ateca.
To create such a flying feeling using live-action 360° cameras was a real test. We initially discussed ways of doing this in full 3D, but in the end the visual representation of the city had to be real — and we felt like live-action footage would make the 4D experience that much more realistic, when you’re sitting in the chair synced-up with those camera’s movements.
We filmed in the most iconic locations in Barcelona, using custom-built stabilized high-end 360° camera rigs, made with GoPros as well as Sony A7 DSLRs. We mounted those rigs into drones, backpacks, and used them to create seamlessly smooth moving skateboards shots that give you the feeling you’re literally flying, using a skateboard’s natural bendiness for steadiness.
What I really wanted to capture in this film is the day-to-day spirit of Barcelona, it’s people, and the energy of this amazing city. I wanted to create a journey that captures Barcelona’s magic, and keeps you at the edge of your seat throughout. So we had to find a balance between light and flexible equipment, and a more polished, high-quality setup.
Experience vs. Chaos
I’m always fascinated by the different skills that are needed to create complex virtual reality projects. On one hand, you have the traditional film crews. They are highly organized, and specialized in their field. There’s very little overlap in responsibilities, everyone knows what they need to do, and their strength is in repetition — applying their experiences over the years to new situations.
On the other hand we have the innovation teams that build new 360 camera rigs, and who improve and embrace chaos as part of the process of creating virtual reality work. Their skills are exactly the opposite.
They are comfortable with chaos. They don’t have defined skill-sets, and responsibilities overlap. They are comfortable with the insecurity that comes with innovation work — in other words: not knowing exactly how to solve a problem, but being ready for anything.
As a director, it is not only my task to come up with the story, and to set the scene for each location, or to discuss camera moves, or work with the actors. I also act as a bridge between these two different mindsets, experience vs chaos.
Car & Camera — A Collaboration
The camera work required to make this project possible was extremely challenging, especially considering the complexity of 360° filmmaking.
To make all of that possible, it took the brains and skills of one the industries best camera teams, under the leadership of Director of Photography Carl Burke.
But since the camera was always moving, Carl worked closely with our lead actress, stunt woman, and hero driver, Natasha. Together they created a smooth interlinked choreography, almost like a dance, between the camera and the car. Making those two elements connect took careful planning, and a bit of luck.
The Hardest Shot
The opening shot of the film is an epic drone shot, flying high over the sea towards the Barcelona shoreline. The function of this shot is to establish the city skyline, but as the drone dips closer to the water, up ahead you see a man with a jet board rising up out of the water.
This was our most complex shot of the whole shoot. We had only about three hours to shoot it, because of the specific ‘magic hour’ look we wanted. The interplay between the drone, the jetpack, and the two jetskis coming up from behind, meant three moving elements all needing to align perfectly.
The best teams are the ones that give you flexibility.
We spent about an hour the day before the shoot on the water getting a sense of distances, and deciding on positioning. Once we were ready to film, at about 5am the next morning, we kept getting the timing between all three of those elements wrong: sometimes the drone would be too fast, another time the jetskis would be triggered too late, etc. We had so little prep time, that we were all improvising, hoping to get it right.
At the start of the shot, the drone was flying about 5 meters above the water’s surface, at speed.
With constant tweaking we finally managed to get a perfect take around the 8th try, and then after a few more runs, had to move on to the next setup.
It takes a very experienced and disciplined drone team to pull of a shot like that — and to do so while maintaining a safe work environment. After all, drones are dangerous objects.
That environment allowed me to go with the flow, to improvise on my creative vision, and adjust my plan on the fly. This is so crucial, the best teams are the ones that give you flexibility.
In the end, I think this shot is the perfect opener, something which establishes the viewer point-of-view, and sets up their movements and speed throughout the rest of the film. It creates a sense magic right from the first frame.
Shot 7 in our film took place on the famous La Pedrera rooftop. This is a building designed by Gaudi, an historic monument, which meant we had very limited access. We would have over 40 actors, and a technocrane, as well as all our normal crew and equipment. But we could only access the rooftop at midnight, and would have to be wrapped and out by 8am, the next morning, to allow tourists to enter as they always do, during the daytime.
So what would feature as a sunset shot in our film, was in reality shot during sunrise.
Because of the low-light conditions, we build a special 360° camera rig that incorporated 4x Sony a7 DSLR cameras, allowing a much higher picture quality, and giving us the opportunity to start filming when it was still dark. We also added a GoPro rig, in the nadir of the main A7s, as a backup. But we were really hoping we wouldn’t need them.
Apart from the challenge of rolling shutter, shooting with less cameras like we did also has a knock on effect, that you can’t get too close to your actors. And since this scene was meant to feel like a rooftop party, we really wanted the cast to crowd around and fill up the area around the camera as much as possible.
Our cameras we suspended from a 40ft. crane, which would start dipped low, and then travel through the crowd, to the other side of the rooftop, in a smooth arc.
The location is stunning, and slightly haunting, with these huge heads towering over you. The sunlight, first deep red, then orange, and gradually, as the morning wore on, fading to pinks and blues, was mesmerising.
I always have a lot of respect for the cast members of all of my shoots. They are people with an amazing amount of patience, and dedication. On this particular setup, they went above and beyond. Having a party at 5am in the morning, and looking lively, excited — as well as repeating the same actions perfectly, while we went through about 20 takes, is harder than it looks.
Post-Production & VFX
The Post-Production work was also a very detailed and meticulous process — removing all cameras, equipment and shadows, as well as Carl himself from the footage. Under the leadership of Zlaten del Castillo, and Kassi Koufaki, UNIT9’s in-house post-production team set up a separate floor, with a dedicated render farm, to make all shots possible. Our compositing team would stretch to 10 Nuke artists to get all the work done on time.
Our team worked using a combination of Nuke and After Effects, as well as a series of custom coded plugins, to stitch and stabilise the footage we shot, and clean up each frame, at times one-by-one, frame by frame.
Here’s a downward facing view of Carl, on a longboard, following one of our main skateboarders.
The post-production team mapped the floor using photogrammetry, and tracked it into place, taking into account not only the floor texture, but all the imperfections and the way light changes as the shot progresses.
Equally, some of the high-speed driving shots were a real challenge. We filmed one shot along the main road in front of the harbor, the famous Passeig de Colom, winding around a statue of Columbus pointing at the New World.
We filmed this with a 360° helmet cam, with our rider trying to keep his body smooth and steady, but also as small as possible. The post-work involved a lot of stabilising, as well as meticulously painting out the bike and it’s shadow.
I find it interesting to observe all of these post-production process. They are incredibly complicated, but software is getting better all the time and many of these steps are slowly becoming automated.
Many of the technical challenges of today, will be automatic tomorrow.
One of the most interesting creative challenges on this film were the transitions. It is impractical to make an experience without cuts; if you’re travelling through a city it could take an hour to drive from one area to another. So transitions would be vital in making sure the viewer can cover more ground.
I worked with Gareth Blayney to develop a stretched-tunnel effect. This would be easy to do in a tradition film format, but in our case, the tunnel would stretch all around you. And the stretch/blur we applied to your vision would need to connect smoothly, without creating visible stitch lines.
In addition to this, we wanted to map shapes from the first shot, to similar shapes on the next one. As you can see in this example, we mapped sections of the buildings around you, and morphed them to try to create a visible symmetry between in the whole sequence.
Music & Sound
Fede Pajaro is one of the most experienced sound designers we could have hoped for. But as 360° degree filming implies, the challenge is always where to hide the microphone. On every location, Fede and his crew had to find clever tricks and smart solutions to make sure they could record a clean stream of audio, with either an ambisonic microphone, or multiple lapel mics, positioned in four places to provide quad coverage.
The real work with sound design starts once the edit is done, however. We spent about two weeks working with the final edit, to create a fully ambisonic sound mix that takes place in all directions around you, and which mixes in many sounds recorded on location, with additional layers of mood sounds to build up a complex, beautiful soundscape.
It is frustrating that most VR players on the market do not allow for ambisonic sound to be played properly. It takes a very powerful setup to experience the true dynamism in the sound design work from Fede.
The music, from the legendary Nina Simone, was of course, the perfect platform to work upon.
Paris Motor Show
This Virtual Reality experience was premiered at the Paris Motorshow, allowing visitors to experience this thrilling ride in 4D seats, which were programmed to move with our camera-work as the film plays.
SEAT VR CREDITS
BRAND Seat Made with Ogilvy Spain, Wildbytes Driver Natassia Duran Shoot — UNIT9 Films Director Anrick Bregman Producer Holly Restieaux DOP Carl Burke Music Nina Simone
Like this story? Get more in the weekly newsletter from CinematicVR.