Fifteen Unconventional Uses of Voice Technology
I recently taught a 7-week long, Master’s-level course at NYU’s Interactive Telecommunications Program called Hello, Computer: Unconventional Uses of Voice Technology.
The goal of the class was for students to learn the technical skills to work with emerging voice technology, and use them to make artworks, games, apps or installations that were outside of the practical ways we currently talk to computers.
Students had half a semester to learn tools like the Web Speech API, Dialogflow and Actions on Google, and then were tasked with making something…interesting. The in-class code examples we used are on GitHub.
Here are fifteen funny, subversive, and impressively weird final projects from the class:
Sad Bunny is a digital representation Kimberly Lin made of herself, in the form of a sobbing, sniffling Google Home who you console through a recent break-up. Kim reverses the emotional labor dynamic between human and voice assistant, making you patiently listen to the intimate details of Sad Bunny’s romantic experience and give her advice.
Itay Niv made BeatBot, which turns your typically dry computer-speak into sick beats. It’s a web-based speech synthesis sequencer that even raps on top of itself.
Yiyao Nie made the AR pet she wishes she could have in real life: a dog that listens to what she says, understands and reacts, responding to commands such as “sit” or “come” or “lay down”. While we still can’t cuddle virtual pets, shouting and having them come towards you from across the room might be the next best thing.
Inspired by the experience of drinking soju with your buddies in Korea, Joohyun Park made Mystery Number, a version of a game where everyone has to guess the number that’s etched into a bottle cap. Appropriately, the computer voice you’re playing with sounds drunk.
Mengzhen Xiao created the Geegle Assistant, which is very much like the Google Assistant, except for one thing: it has feelings. The Geegle Assistant can do all the normal functions you’d expect in a voice assistant, but it has a strong personality and isn’t shy about expressing disappointment in your behavior.
Maï Arakida Izsak uses Twitch comments to create a feeling of co-presence for someone in a VR headset. In VR Voicemail, Maï streams her scene in Twitch, allowing viewers to comment in the stream chat in real-time. When someone leaves a comment, a pink bubble falls into the scene. When Maï touches the bubble, the comment is read out loud in speech synthesis, and broadcast out again to the stream.
Aidan Nelson made Paradise Blues, a voice-controlled VR experience that makes you the god of your own personal hell. You say things like “Bring me 50 chickens!” or “Let there be refrigerators!” and the objects you summon rain dramatically from the sky until they pile up in a mess all around you.
Jenna Xu created an assistant to help in her own life, featuring a guided breathing exercise, guided physical exercise, guided cognitive behavioral therapy, and a gratitude log. By making a tool that intervenes with life’s common problems, Jenna uses Life Support to turn her self-care into a regular practice.
Ilana Bonder wanted to make something functional and genuinely useful for her daily life and the life of her fellow students, so she created the ITP Assistant. The ITP Assistant allows student to find a teacher who can help with specific topics, and then books office hours with them.
Makeup Guru, by Alice Sun, generates a fresh eyeshadow makeup lewk based on a color of your choice. The colors themselves are selected in an unusual way — from word vectors created from sorted RGB values. There are also some unconventional makeup colors in there, including “poo”, “baby puke green”, “snot shade”, and “diarrhea”.
Alden Jones created Leaf Town, a device that lets him talk to his plants and hear them talk back to him. It’s a speculative fiction piece that imagines the inner life of his plants, who have to be wired up, because obviously plants can’t communicate using wifi and need to be connected by their roots.
Nick Wallace, like many of us, needs a drink after a long day. He built Virtual Bartender, which asks you for your liquor of preference, and stirs up a cocktail recipe for you.
Shreiya Chowdhary created Wee Wee Piggy an twisted voice game version of the nursery rhyme “This Little Piggy.” You use voice commands to move the pig around the page, trying to get him home, but instead of saying “up, down, left, right,” you have to say “sausage, bacon, ham, chops.”
For those of us who have suffered heartbreak, Ridwan Madon made Breakup Aid, an Google Assistant Action that helps you get through and recover from a breakup. The app is meant to be a companion that may be easier to talk to than another human being when you want to vent.
Keerthana Pareddy made Feed Me The Buzz, which is her voice-only version of the ubiquitous Buzzfeed personality quizzes. Using generative text techniques, the app asks you 10 questions, ultimately then telling you the next fake Buzzfeed article you should read.
