Case study: Mikko Joensuu 360° music video
Director’s take on how a group of unexperienced students delivered a beautiful 360° music video, with very limited budget.
In autumn 2016 got a chance to direct a 360° music video for Finnish artist Mikko Joensuu. This was a pilot project between Finnish Broadcasting Company YLE and Metropolia University of Applied Sciences. Our team was commissioned to create a 360° music video for yle.fi/musiikki and in the end more than 30 students from 3 different campuses were involved in this production.
For most of the crew it was their first attempt to create something for this new and fascinating medium. Luckily we received wonderful guidance from visiting lecturers Synes Elischka, the director of VR film Ego Cure, and Serdar Ferit & Paulina Tervo from Lyfta. They already had a lot of experience of creating 360° and VR projects.
As a director I felt urge to explore the medium by myself first. Later on, with my producer Iisa-Noora Leppänen we managed to find right individuals to the key roles of this production. They did absolutely great job studying, exploring and sharing their knowledge and findings as the production went on.
From my point of view this was a zero budget production. Of course we had a great bunch of labor and access to some expensive gear and a studio space. “No-budget” was still a limiting factor while we were planning the video.
First, find a location that works for 360°. Then define the action in that specific location.
Oftentimes limitations are actually good. Lack of money forced us to find real world locations and craft the story to fit into them, which suits perfectly for 360° storytelling. We knew there would be a great bunch of people ready to do motion effects and post production. That’s why we decided to concentrate on capturing the best possible canvas for them both in locations and studio.
Shooting 360° can be fast!
(just remember to plan well beforehand)
The whole video was shot in 3 days. We built and lit the studio on Wednesday and shot it on Thursday. On Friday we shot two location scenes. Beach location was shot later in one afternoon, it’s actually shot in bright daylight.
In the studio we shot five takes total, with ten musicians playing the song live. Audio was recorded in all takes and the studio take is actually selected based on the live audio rather than the visuals.
In road and courtyard scenes we had a small location crew and two actors. The mysterious lady is Finnish professional actress Marja Salo. She wanted to participate our project after giving us a lecture about directing from actors’ point of view. The guy in the courtyard is Severi Haapala who works in Metropolia.
What more should I say about the video? It’s an experience which works best with a VR headset and headphones. There is a loose storyline, which in my opinion relates to Mikko Joensuu’s whole Amen trilogy.
We wanted to create a peaceful space, where the viewer can listen to the beautiful tunes and immerse into another world.
That said, if you liked the song, you should definitely try it with a cardboard or a headset. It’s totally different experience than viewing it with a desktop viewer.
Medium driven creative choices explained
Usually I don’t want to explain creative decisions that much. This time I’ll make an exception, because I feel it’s really essential for understanding this new, emerging medium.
Rather than telling a clear, narrative story we wanted to provoke universal feelings in different scenes, both emotional and physical.
Courtyard location was selected because we wanted the viewer to feel trapped. Perfectly round courtyard is such a cool location for a 360° video anyway! Severi’s character gives the viewer a clue of their appearance and state of mind. This is one of the challenges especially in 360° video. It’s really important that viewer can identify themselves right from the beginning.
As a viewer of live action 360° videos your interaction with the scene is mostly limited to just turning your head around. All other interaction with the virtual world must be channeled through other characters.
First, the male character picks up some papers and desperately tries to keep them in order. This gives us a clue what we might be doing if we’d be able to move. Then he opens the gate, which also is something we are not able to do. Talking about physical immersion, while the gate opens and fog starts to roll in there is a brief moment when viewer might actually feel like floating.
360° doesn’t mean that you should have action all-around, all the time!
Road location was selected because this scene needed to have just two directions. We wanted to give the viewer this feeling of letting someone down. Character goes through three different mental states in this scene. At first she is very confident and arrogant. After a while there is a short moment of disbelief. In the end she breaks down completely.
Nailing this scene sounds pretty simple but it wasn’t. As you see there is no place to hide anywhere nearby. Me and the camera crew were hiding in a ditch about 30 meter away so I had no way to monitor the scene.
We talked about the scene with Marja before the actual shoot. At the location we rehearsed few times so that I was standing next to the camera. When I felt confident, we just put camera rolling and went hiding. There was a walkie talkie hidden nearby the camera so I was able to give directions. There I was, laying in the ditch watching a timer clock running, speaking to a walkie talkie and hoping for the best. Pretty glamorous!
While we were discussing about the road scene, our actress kindly guided me to think more like a theater director. Their work is pretty much done by the time the actual play starts. This is often the only way to work in low budget 360° productions.
That said, I don’t have that much experience of directing theater or cinema. It can also be a good thing while exploring new stuff. Even if you are a seasoned director or producer, don’t take anything for granted. In this production I had to learn so many new things. Some simple & effective filmmaking conventions might also need a bit of re-thinking before they can be used in 360° video productions.
1 roll, 2 roll, …, 6 roll. All cameras roll!
Drop me down 360° was shot with 360Heros Gopro rig. The rig has six Gopro Hero4 blacks aligned diagonally against each other. Working with this kind of setup certainly isn’t an easy workflow.
There are six separate cameras to take care of. If one of them fails, your image will have a black hole somewhere.
Then on the other hand, if one camera blacks out, it’s pretty easy to replace it with a new one and keep shooting. This actually happened to us during our shoot. I got home after a long day in studio and did a camera check just before bedtime. We had a shoot scheduled very early next morning. Of course one of the cameras didn’t turn on at all!
After some moments of panic and cold sweat I got myself together, removed the battery and voilá, camera was working again! If one of the cameras fail in high end cameras like Nokia Ozo or Insta360Pro you can pretty much call it a day and start looking for a replacement.
Good practices while shooting with a Gopro rig
Operating a Gopro rig demands some time and discipline in set. There must be a way to charge batteries in between the takes. In remote locations USB power banks work well. If there is AC power available, a powered USB hub with long cord is great. Bring the power to the camera, not the other way around.
Once the camera is set up it should stay in that exact place until that scene is wrapped. This can be a lifesaver in some cases, because you might be able to combine footage from different takes.
Correct settings are really important. This Gopro rig is designed to work with 4:3 image ratios so there is only two resolution options. If you accidentally record in 16:9 ratio, stitched image will have black triangles everywhere! I’ve seen that, not a pretty sight…
When resolution and frame rate is set correctly, next thing is to try to lock all the other parameters. It’s a good idea to lock WB and set ISO limit to 400 ISO. Actually exposure is the only thing which cannot be locked, it can only be compensated +/-2EV.
+/-1 EV compensation is useful if some of the cameras are facing to darker areas or intensive light sources. If you try to combine two images which have more that 1EV difference, there will be some nasty color fringing around the areas where images are stitched together.
In controlled lighting environments there is a workaround for getting the correct lighting parameters. Gopros have constant aperture of f/2.8, minimum shutter speed in 50p is 1/100s and when ISO limit is set to ISO400. This way it’s possible to control the lighting especially in the darker areas of the scene. I wouldn’t go over ISO400 since the noise is pretty horrible in higher ISOs. In our production cameras were set to record 1440/50p, Protune on, color flat and ISO limit 400. We could have also shot in 2.7K/25p for higher resolution and better lowlight, especially in the studio.
Nailing the camera setup perfectly is essential! Take your time, check and double check before the shooting starts. If you must hard reset one of the cameras during the shoot, check all settings again!
Always start cameras in same order. This helps later when you are syncing and stitching footage. With older rigs you cannot start cameras simultaneously but you should try to start cameras as quick as possible.
First press Rec in all cameras, then make another round to check that they actually started recording.
Take your time but try to keep your takes as short as possible. Cameras run out of power pretty quickly and six cameras recording high frame rates produce a great amount of data.
Using a clapper board or a clicker is essential while recording high frame rates. Avoid any other sounds or commands 10 seconds before and after the clap.
Keep your data organized after the shoot. If you remove cards from the cameras you should still keep track which card is from which camera. It’s also a good idea to rename files before post processing so that you know where the footage came from. Organizing and renaming must be done manually but it’s definitely worth it.
It’s time to stitch — keep your fingers crossed!
Stitching means combining the images to a panorama. I our production it was done with Kolor Autopano bundle. We made rough stitches during the shoot to confirm that images were usable. This might sound time consuming but it’s very important, especially if you’ve used a lot of time and effort to create the scene.
Don’t move on before you’ve confirmed the stitch!
360° live streaming with Videostitch Vahana VR helped us a lot while we were building and lighting the studio set. Actual takes were shot using the Gopros’ internal recording to keep the camera footprint minimal.
Final stitched scenes were huge 5.5K/50p 10-bit Cineform files. Later we had to drop the specs to UHD/25p due some unexpected troubles in the post production pipeline.
Visual effects wizardry
The story got a whole new layer in post production, as you can verify from the making-of clips. Super talented and motivated group of students worked some serious hours to make this happen. Most of the post production was done in After Effects with Mettle Skybox plugin. Maxon Cinema 4D was used for creating the 3D effects.
Originally, we didn’t want to move the camera because of the fear of VR sickness. This decision also gave us much more room to play in the post.Otherwise it would have been impossible to rotoscope and create depth masks inside the production time frame.
Special thanks to Lauri Huikuri and Johannes Metsälampi, who took charge of the post production. Thank you Liisi Soroush who helped me to define the visual aesthetics and supervised the visual integrity throughout the production. Thank you senior lecturer Minna Kilpeläinen who made this possible in the first place, and thank you everyone who stubbornly made this happen! Complete crew list and credits are attached to the Youtube video, too many names to mention here.
If you want to ask something, don’t hesitate to contact me. I’m more than happy to share my knowledge!
