Behind the pixels with Scott Millar and Lewis Kyle White
Launching new music is a tough gig for musicians in COVID times. With regular festival runs and arena tours currently off the table, artists are looking to the internet to promote their albums and connect with their fans.
Kat Kemsley, host of the Notch Showcast, sat down (virtually, of course) to talk to Scott Millar (XR Studios) and Lewis Kyle White (Pixels & Noise) about their groundbreaking work with Block 9 on the Gorillaz live streaming concert performance.
Live stream performances have been the alternative chosen by many artists, but these concerts come with their own set of challenges. Without an in-studio audience, how do you communicate the buzz of a live performance? What separates a live stream performance from all the pre-recorded media already available? How do you make the at-home-audience feel like they are a part of something irreplicable?
Block 9 saw these challenges as an opportunity to create a unique ‘tranzmission’ that would unite the Gorillaz with their fictional, mystical counterparts. Live-streamed over three days to Europe, Asia, and the U.S., the concert featured the Gorillaz — in both human and animated form — performing with guests beamed in via hologram.
Block 9 collaborated with XR Studios and Pixels and Noise to deliver the show’s technical workflow and real-time design. Drawing on TV references from the late 1970s, the team used virtual production techniques to blur the line between real and imagined worlds seamlessly. The result is a strikingly outlandish and sometimes bizarre mixed reality show that galvanized at-home audiences worldwide.
Where did the inspiration for the Song Machine live stream originate?
Scott: “We were approached by Block 9, who led the production and creative direction to help them create a mixed reality live-stream. We had worked together on Glastonbury’s IICON stage in 2019, but this production would be quite different.”
“The difference between a live stream and a live concert is that we know that the audience watching the live stream will be watching the screen. We had this unique opportunity to fully control what the audience sees and transport them to the world of the Gorillaz — in this case, the vast ‘basement’ of ‘Kong Studios’. Gorillaz is a band epitomized by their legendary animated counterparts designed by Jamie Hewlett.“
“Fans love the comic book fantasy world of the Gorillaz as much as they love Damon Albarn, so we saw a unique opportunity to combine the two using virtual production techniques. We also used these techniques to solve a current problem many productions are facing — travel and the number of persons allowed on-site is limited. Using AR, we were able to bring featured performers Beck, Fatoumata Diawara, JPEGMAFIA & 6lack into the live performance space. Block 9 and Pixels and Noise found some clever ways to have fun with it.”
Do you mean transforming them into 70’s sci-fi holograms?!
Lewis: “The Hologram Treatments were a lot of fun. We found subtle ways to mess with the traditional hologram idea, like adding data moshing post-processing effects to corrupt and distort JPEGMAFIA’s hologram feed. For Beck’s hologram, we took inspiration from the Elvis scene in Blade runner 2049. We used an RGB glitch to make it look like the light from the ‘ holographic projectors’ was breaking down. And then we keep it super simple and ethereal for Fatoumata. It was nice to carve out these narratives at every level of the creative process.”
How did you achieve this technically?
Scott: “Performances from the hologram guests were filmed at separate locations on a green screen, which was then chromakeyed and treated with effects by Lewis. When we were on location, we took the positional data from our six tracked cameras and the live video feed into Notch where we could add the keyed footage as AR elements on top.”
I noticed some interesting effects on the stream, making it feel like a music video; how did you create these?
Scott: “I think it has this music video feeling from being live vision mixed, cutting between six different camera angles creates a lot of energy and dynamism — and of course the live IMAG effects created by Lewis.”
Lewis: “From the outset, we followed a creative set out by the band’s acclaimed animated music videos, focusing on the collage-style approach of the most recent videos. We started with subtle film effects, but once we got going, things quickly ramped up to ‘let’s make this section look like an Acid trip as Murdoch has just drunk some poison!’ Ultimately we had a lot of freedom to experiment with IMAG effects and create looks that ranged from subtle to extreme. The creative, flexible nature of the Notch workflow produced both carefully crafted results and happy accidents.”
Scott: “Layer composition was another important part of our workflow, and being able to easily switch the order of layers helped us generate a lot of variants on core looks. It was great fun being able to mess with the program feed in this way — it helped us achieve something that looked original and a bit strange at times.”
How did you bring all these elements together?
Lewis: “Notch allowed us to handle everything all at once, from the 2D in the 3D tracked environment to shadow catchers, floor reflections, and virtual lighting rigs. It was an active collage of events and creativity that all came together to produce a tailored visual experience.”
Were there any challenges you had to overcome?
Scott: “Like a lot of productions, we’ve had to adapt the way we work to meet COVID safety requirements. During the build, everyone worked remotely, so we set up a VPN network to connect everyone’s machines. Notch’s network editing features made on-site edits simple. Lewis was able to edit from his studio in London remotely, and the wider team would be able to see the updates in real-time. We didn’t have to worry about file shares or anything like that.”
What was your favourite part of the show?
Lewis: “This whole project was a fun one to bring to life. The process of uniting the real-life Gorillaz with their beautifully animated counterparts was just incredible.”
Scott: “Momentary Bliss is my favourite part as it’s where things go a bit mental. The band wanted this track to have masses of energy. Using Notch, we were able to re-imagine themes from previous Gorillaz music videos as live-rendered IMAG effects. A lot of these effects we were able to scope out and create during rehearsals where we could build and layer FX on the fly to create a final look that fit the bill.”
What advice would you give to someone looking to try out these techniques?
Scott: “Lighting is key when working with virtual elements and live-compositing. We built virtual lighting in Notch and gave our lighting designers full control. This helped us integrate the AR effect with what was happening in real life on stage and seamlessly blend the characters into the live environment — it was also an efficient workflow for our lighting team.”
Lewis: “Granular details are important, and the attention to detail from Block 9 and the team is evident in the final result. We poured hours of work into this production, so I’d recommend finding tools that help you work fast and flexibly while maintaining your creativity. Having Notch at the heart of our workflow really empowered us to experiment and try things out.”
Thanks to Scott and Lewis for shedding some light on what went on behind the scenes for this very special performance. Both will be guests in the upcoming Notch Showcast Season 4, so look for those episodes in the coming months!
We also want to thank XR Studios, Pixels & Noise and Block 9. Check out their websites for more information about their exciting projects!
