Lights, Camera, AI. From Andy Warhol reading his own diaries, to post-production wonders of the Marvel universe
From Synthetic Dialogue to Restored Footage, AI has established its way in Film and TV Production in the recent years and for the next to come.
The 6 episodes for the documentary series ‘The Andy Warhol Diaries’, chronicling the life of the late artist Andy Warhol, was released in March 2022. The six-part Netflix series executive produced by Ryan Murphy (American Horror Story) and directed by Andrew Rossi uses AI (artificial intelligence) technology to recreate the voice of Andy Warhol reading his diaries. The voice of the American pop-art pioneer was created using a text-to-speech algorithm that incorporated his native Pittsburgh accent. Actor Bill Irwin then recorded the sentences, with his performance and the digital AI voice combined to create the voice heard in the documentary series.
“Andy Warhol famously guarded his personal thoughts and opinions. This is one reason his diaries are such a rare and fascinating window. he could be incredibly tough and emotional as he spoke to his diary manager on the phone,” Andrew Rossi pointed out. “To fully appreciate the extreme sensitivity Andy reveals in his diaries, I felt we needed to hear the words in Andy’s own voice,” he added.
The AI system used in the docuseries is based on a neural network architecture called WaveNet, which was developed by researchers at Google’s DeepMind. WaveNet is a deep learning model that can generate high-quality audio waveforms by modeling the raw audio signal directly. In the case of the Warhol project, the system was allegedly trained on over 1,000 hours of audio data to create a realistic and authentic sounding voice for the artist.
The use of AI to recreate Warhol’s voice has generated mixed reactions from fans and critics alike. Some have praised the technology for bringing the artist’s voice back to life and allowing viewers to experience his unique persona and artistic vision. Others have raised concerns about the ethics of using AI to create new content from the voice of a deceased artist, and the potential for the technology to be used to manipulate or distort historical records. Filmaker James Cameron, has been vocal about his concerns regarding the use of AI in film production. “I think we need to be careful not to let AI become a substitute for human creativity and judgment. While these tools can be incredibly useful in certain contexts, ultimately it’s the human touch that makes great movies and television shows. We need to ensure that AI is used in a responsible and ethical way, and that it doesn’t undermine the integrity of the creative process or the diversity of voices in our industry.”
Dispite concerns, AI has been previously used in the movie industry for experimentation and optimisation and has established its way as the new norm. In the film “Sunspring” (2016), conceived by BAFTA-nominated filmmaker Oscar Sharp and NYU AI researcher Ross Goodwin, the entire screenplay was generated by an AI algorithm trained on science fiction scripts. Director Aaron Sorkin also used an AI system called Cinelytic in “Molly’s Game,” to analyze audience data and predict the optimal release date for the film. The system also helped the production team to identify which scenes were likely to be most emotionally impactful for viewers, allowing them to focus their resources on those parts of the film.
The recent Beatles documentary “Get Back,” directed by Peter Jackson, also utilized AI technology to restore, clean and enhance the original footage and sound from the sessions. The documentary was created using footage from the recording sessions of the Beatles’ “Let It Be” album in 1969, which had been previously released as a film of the same name. The result was a stunningly clear and detailed picture that brought viewers closer to the original recording sessions than ever before.
Dispite the technical advancements, the true power in AI lies in what value can add to the creative process, rather than anything else. And it is yet to demonstrate that AI can excel in producing time-lasting thought provoking masterpieces, but rather slightly boring, semi-procedurally generated stylesets. However, the ability to restore and bring to life characters and information that has been lost during the years is equally magical to a timemachine. And in the end, even in ways that are so subtle that can not be detected, creativity is about inspiring the magic in the real world. “Beginning with his childhood in Pittsburgh, the series follows Warhol’s almost unbelievably diverse journey moving between media and eras as an artist and filmmaker, publisher, television producer, celebrity who was recognized but also the target of criticism” is stated in the description of the documentary.
Andy Warhol passed away on February 22, 1987, in New York City, United States and is one of the most influential artists of the 20th century.
