IMMERSIVE EXPERIENCE
What It Means to Be Human in the Age of Artificial Intelligence
Luxembourg Tech School’s version of FrankensteinAI
FrankensteinAI is an immersive, interactive experience developed by Luxembourg Tech School (LTS), based on a multi-year research project of Columbia University’s Digital Storytelling Lab. It aims to explore the social and ethical implications of how we design and interact with Artificial Intelligence (AI) and what it means to be human in the age of AI.
“Knowing the future is impossible. But what we can do: Knowing what future we would like to have. And then work on it.”
- Have you ever thought about what makes you human?
- What makes you different from others or from an intelligent machine?
- If we create something like an app or a robot, are we responsible for its consequences?
Artificial Intelligence (AI) might have the potential to help us find answers to problems we have not been able to solve yet. For that reason, we need to discuss and demystify AI. Looking from different angles might help us to overcome commonly known perspectives, fears and desires and provoke conversation around possible AI futures.
What might happen if we invited AI to talk with us?
FrankensteinAI is a multi-year research project developed to explore the social and ethical implications of how we design and interact with AI and is being carried out in various forms around the world. It has its origins at the Columbia University School of the Arts’ Digital Storytelling Lab / lance weiler and premiered in 2018 at the Sundance Film Festival.
Students of the Luxembourg Tech School worked over the course of 7 months on an own version of an immersive experience and presented it for the first time as part of an off-beat 20th anniversary evening event of the Fonds National de la Recherche Luxembourg (FNR).
FrankensteinAI: The Event
In November 2019 citizens and scientists came together in an old townhouse in Luxembourg City to immerse themselves in some of the biggest controversial science topics facing society, such as Artificial Intelligence (AI), genome editing, gender balance and the use of space resources.
In the Mary Shelley room, guests walked in to see a cube on a table. The cube — called Frankie — was the ‘mouth’ of an Artificial Intelligence, connected to an AI in the cloud.
Frankie talked to the guests, explaining that it has learned that humans are social creatures, and that it could not understand humans by just meeting them online. Frankie wanted to learn about human emotions: it asked questions and encouraged the human guests to take a critical look at their thoughts, hopes and fears around technological innovations. To question stereotypical assumptions and share their feelings and thoughts with each other.
When leaving the room, the guests received a self-created handcrafted paper booklet with further content about AI, Frankenstein and the whole project.
The experience gives food for thought — both about the increased digitalisation of our world, and way of communicating with each other, while also giving a taste of how AI may not feel emotions, but can “read them”, prompting many questions. It raises the question of responsibility we have towards scientific and technical achievements we create and use. Mary Shelley’s Frankenstein novel presents a framework for narratively examining the morality and ethics of the creation and creator.
Event Design Process
The entire project is based on a human-machine collaboration.
We aimed to dig into the tensions between human and machine in an immersive environment that mixes visuals, sounds, space, narratives, technical tools and conversation.
In order to build our creature and to make it grow, we of course had to feed it. This was achieved by labelling every sentence of the Frankenstein novel with up to three of nine basic emotions (curiosity, surprise, love, trust, sadness, anger, disgust, fear and joy).
We then used the labelled sentences with the same emotion to feed a Machine Learning model and generate new paragraphs of text in the style of Mary Shelley’s Frankenstein novel.
We used adapted excerpts of the original novel to ask how our creature should feel in certain situations. To create the questions, we filled a few pads of sticky notes with personal questions pertaining to the nine basic emotions. We then fed the questions for each emotion into the same model to obtain additional and sometimes eerie questions.
We wrote a whole script, where Frankie (the AI) and human narrators guide people through the whole experience and enriched the experience with self-programmed visual projections and sound.
Technology & Tools used
Our most important tool, the base of the project, is of course the original novel “Frankenstein; or the Modern Prometheus” by Mary Shelley. However, we were lucky enough to get our hands on a special edition released by the MIT Press: Frankenstein: Annotated for Scientists, Engineers and Creators of all kinds. This edition includes relevant and concise annotations, that help with understanding the scientific and historic context, and thought-provoking essays.
Processing by the Processing Foundation is a free open-source Java-based programming language that makes creating impressive two- and three-dimensional graphics easy. We used it for creating animations around the nine emotions.
RunwayML is a powerful but accessible machine learning tool that allowed us to run models remotely and provides a simple graphical interface. We used the models GPT-2 by OpenAI and Photosketch.
Google AIY Voice Kit is a small cardboard box through which guests heard the voice of the creature/Frankie. The kit allows to assemble a Raspberry Pi –based smart speaker, which can be connected directly to Google Assistant, or to program it to use Google’s text-to-speech engine to build an own speech-based application.
FrankensteinScript is our own self-developed system for managing the event flow. It is based on scripts that control what the Voice Kit says, when to listen or what to show on the screen. The system works around a central server that receives information from the different devices and controls them accordingly.
Discuss with each other. Discuss with us.
We would like to move the conversation around Artificial Intelligence away from fears and build awareness around the ways in which AI can be used as a tool by humans to accomplish all kinds of things — good and bad.
- What benefits do you see in AI applications?
- How strongly is your everyday life already influenced by AI?
- Do you know examples in which AI systems could do some harm?
- Shall the development of AI systems be regulated? If yes, by whom?
- Will intelligent machines replace us, coexist with us, or merge with us?
What will it mean to be human in the age of AI?
Our version of FrankensteinAI was co-developed by LTS students Barbara Hougardy, Henri Ahola and Nora Dieschbourg under the lead of LTS Program Director Sara Kaiser and LTS Founder Sergio Coronado.
Luxembourg Tech School (LTS) is an extracurricular school program to support the development of future Digital Leaders. It is aimed at 11–19 years old students, who are eager to learn and apply technology in a real business context. Currently LTS started to offer first paid job experiences for its students through its “Learning to Work” program.
