Et Moi 360° Move with cuts and grade with AI.
Hi! My name is Herman. I ́m a Swedish VR Director. I first tried VR in late 2014 and was blown away... like everyone else. And have been totally focused on VR since then. My main focus is experimenting how the VR scene will develop. I am especially interested in translating film techniques into VR.
Last summer I made a 360° documentary about vikings and what I believe is the world’s first book trailer in 360°!
I learnt a lot, and used that for my next project, which I made this summer in Berlin. “Et Moi” is a 360° short film. It was part of the official selection at the Kaleidoscope Festival in Berlin on the 16th September.
The Story
I want to show you behind the scenes challenges, thoughts and the process of making this movie. The film tells a story about a couple from the past dancing through memories in Berlin. I wanted to tell a love story using music and dancing — and I wanted to use memories as a theme.
Music
After writing the script out in a short, simple way — as a guide — the first step I took was looking for music. I found a song on youtube that I liked and asked the artist for permission to use it in my film. The name of the song is “Et Moi” by Nym and have 4 million views on youtube!
Locations & Performers
Next I started looking for locations in Berlin and found 5 spots that I liked. I then made up the details of the story based on the locations I found. I also got lucky, I found 2 tango dancers that,
wanted to be a part of project that would work as main characters. I saw them performing in the street as part of a group, and I approached them to ask them for their help.
Camera
One of the techniques I wanted to work with in this film is to make the viewer feel like they can explore the space — like they are not just static. But at the same time of course, I didn’t want to make them confused either.
The film has always five same exact camera positions on every location. Camera positions have different heights to experiment with scale. The dancers are doing the same choreography on each location which makes it possible to cut from location very fast as the dancers functions as a point of reference.
“The cuts should not be disoriented to the viewer if they follow the dancers.”
Grading with Artificial Intelligence
As a final touch I came up with the idea of grading the film through an ap called Artisto that uses Artificial neural networks to create filters. Artisto works like the prisma app but it can also process 10 sec of video instead of just pictures. So I cut up the final film into small sections and let the code do it’s magic — the results were very surprising!
Conclusion
A.Be prepared for the unexpected.
Batteries tend to overheat when used during a long time. I used a Samsung Gear 360 and it happened 2–3 times each shooting day.
You just have to wait 5 min to let battery cool down, but it means you could loose the most important moment in your shoot. Filming is all about capturing something unexpected, the magic moment
But actually the most unexpected thing during shoot was that I also had to be a babysitter since one of the dancers could not find anyone to take care of their baby.
B. Zick Zack move.
If you want to move a character/actor/dancer/baby from A to B you can put your 360 camera in diagonal positions and reframe the point of reference.
“Point of reference is key in 360 film” /Me among others
C. Grading with AI neural networks.
Since Artisto is not optimized for 360 you have to process every 10 sec clips 4 times to get a perfect stitch. One annoying bug I found out is Artisto export cuts a few milliseconds with every clip and that makes your life harder then you try to match footage with music.
D. Software magic and new consumer cameras makes life easy.
One year ago this film would have been next to impossible to make.
- Software like plugin skybox from mettle.com for premier and after effects have features that makes it possible to rotate 360 video to point of interest then you ́re editing(I used this all the time) and much more.
- The gear 360 have easy and fast workflow with nice features like live monitoring (1.5 sec delay) and instant preview of what you shot. The 6 go pro array I used last summer still produce better quality but you ́re totally blind then shooting and takes long time to process.
E. Everything is a remix
I used the basic elements of creativity. I did a lot of research following the fast progress in VR technology and what others have been creating.
During experiments in early 2016 I found out I was not the only one doing experiments. Nick Bicanic tried fast cuts during same period, Anrick had also tried splitscreen and Jessica Brillhart have experimented grade with ai deepdream.
Great inspiration for this film has been picked up from the past. I then transformed and put ideas and techniques I had tried into a combination that became my film, something new.
The documentary “The story of film” had tremendous impact and inspiration to me. We have plenty reasons to look back at the difference from the 1900s to today if “they are so thrilled by the novelty-effect of VR that they forget about the fact that 10.000s of great minds over a 100 years tried to figure out how film ‘works’.
Why do cuts work?
“ Perhaps the explanation are as simple as that: We accept the cut because it resembles the way images are juxtaposed in our dreams.” Page 58 “In the blink of an eye” by Walter Murch.
Moving on into the future of storytelling, I think we still have a lot to learn from the past.
Editing is one clue to narration.
