A.I. Brainwave Opera Has World Premiere at the Estonian Academy of Music
Ellen Pearlman’s new emotionally intelligent, artificially intelligent brainwave opera “AIBO” had its world premiere on February 22nd at the Estonian Academy of Music’s state of the art Black Box Theater in Tallinn.
The opera imagines Artificial Intelligence merging with fascism and love, and asks if epigenetics, or inherited traumatic memory can exist in AI. Human emotions are displayed on a bodysuit of light, and synthetic emotions displayed as red light in the performance space.
In the performance, viewers take part in an interaction between Eva, played by actor Sniedze Strauta, and AIBO, a ‘sicko’ AI. Eva is wearing an Emotiv EEG brainwave headset connected to a bodysuit of light, which was designed by smart textiles designer Dila Demir.
The spoken word libretto between the AIBO and Eva explores the troubled history of their love affair. The audience watches Eva’s EEG brainwave emotional states manifest as colors on her bodysuit. For example, yellow lights indicate excitement, purple refers to interest, green for meditation and red for frustration.
Meanwhile, emotionally themed databanks of videos and music, also driven by Eva’s brainwaves, are projected onto hanging screens and played through speakers throughout the venue.
Eva’s spoken word libretto is converted to text, which is both projected onto a screen and uploaded to Google Cloud. There, AIBO’s response is generated via a custom built GPT-2 AI, created by Pearlman and programmed by Thoughtworks developers Jonathan Heng, Wasawat Somno and Andrew Zhou, with help from technical director Hans Gunter Lock.
AIBO’s response is projected on another screen as text in the performance space, and simultaneously sonified as artificial speech. In addition, AIBO’s response is analyzed for its emotional sentiment, and categorized as positive, negative or neutral. Positive sentiments display as a green light, negative as red, and neutral as yellow.
Further, Pearlman explains that “AIBO attempts to emulate Eva’s projected visual memories to make them it’s own. It wants to become human, not ‘sicko’, but can only create broken video simulations. This allows viewers a glimpse into fractured memories.”
Strauta, performing the character of Eva, continually interacts with audience members. This sets up a live-time feedback loop between Eva’s speech, Strauta’s emotions as analyzed by the brain-computer interface and represented as colors on the body suit, the audience, and the visual and sonic environment.
The opera is based on the relationship between Adolf Hitler and Eva Von Braun. Eva’s libretto is taken from events in Von Braun’s biography.
Pearlman uses the opera to explore two questions: “Can an AI be fascist? And can an AI have epigenetic, or inherited traumatic memory? These questions raise issues pertaining to a potential future where humans and machines may one day merge consciousness.”
How the project came together
In 2019, Pearlman received the European Union’s prestigious Vertigo Starts Residency Laureate Award to develop AIBO. The focus was to create a comprehensible and riveting short performance based on the integration of artificial intelligence and biometrics.
From the summer 2019 through the winter of 2020, Pearlman worked with the Human-Computer Interaction Lab at Tallinn University under the aegis of GoProSocial, led by Aleksandar Väljamäe, the Estonian Academy of Art and the Estonian Academy of Music and Theater.
Technical director Hans Gunter Lock worked with Pearlman to develop a MaxMSP speech-to-text feedback display as a mode of connection between the performer and the AIBO character in the Google Cloud.
Rehearsals and specialized training were conducted with Strauta, including training in contact improvisation. Pearlman then worked with Heng, based in Singapore, Sonmo, based in Thailand and Zhou based in Texas to develop and test the GPT-2 database that formed the backbone of the AIBO character.
The post-performance photograph shows some of the team along with collaborators and audience members. From left to right: Sniedze Strauta, Mauri Kaipainen, Ellen Pearlman, Gerhard Lock, Hans Gunter Lock, Julie Yaunches, Rūta Kruliauskaitė, Andrew McWilliams, Pia Tikka, Tavet Jansen and Dila Demir.
Work designed by Ellen Pearlman with the support of the VERTIGO project as part of the STARTS program of the European Commission, based on the technological elements from the GOPROSOCIAL.
Originally published at https://thoughtworksarts.io on April 2, 2020.
