Organized Vibrations
Designing virtual experiences for the sense of touch
Tur sense of touch is the ultimate sense of space. Our bodies are stages that the world perpetually performs on. With advances in technology and shifting attitudes fueled by virtual reality, we are beginning to unlock the intelligence and expressive potential of touch as a medium for experiential design. As designers, we are faced with a new world of issues and questions about what it means to create experiences for the sense of touch.
At Sosolimited, we recently installed a tactile artwork in the Cooper Hewitt Smithsonian Design Museum’s new show, Senses: Design Beyond Vision. In the process of making the piece, we encountered some interesting and relevant issues around touch and experience, and I’d like to share some reflections on the subject.
Before I continue, let me fill you in on exactly what our artwork is. It’s called Seated Catalog of Feelings and it uses vibrations and text in a simple, yet surprising way. It’s better felt than read about, but I’ll do my best to put you in the seat.
What is a Seated Catalog of Feelings?
You come upon people sitting in unassuming black wooden chairs, with short phrases projected on the floor in front of them and dancing circles of color on the wall behind. You wonder why they are squirming in their seats, looking surprised and laughing, so you sit down in a free chair.
As you sit, you put on a pair of headphones and hear a woman’s voice dispassionately speak the phrase, a bouncy ball on pavement. Shortly after you feel a bouncy ball on pavement, delivered to your body through low frequency vibrations coming through the chair.
If you were to close your eyes and sit in the chair for an hour, you would experience a sequence of over 100 different sensations, ranging from the banal [microwave popcorn] to the fantastic [an avalanche of frozen peas] to the macabre [the last nail in your coffin].
There is also a vibrating pillow, created for people who use wheelchairs, that you can pick up and feel. It works just like the chairs, but you put it in your lap or hug it.
If you are asking yourself, “How do I feel a bouncy ball or getting sawed in half by a drunk magician through my butt, sitting in a chair?”, then you are already getting into the good stuff, the interesting questions around what it means to experience and remember through touch.
Touch is highly suggestible
It occurred to me after the piece was finished that we had created a kind of ambiguous lo-fi virtual reality, a series of experiences reconstructed through the sense of touch, made possible — and compelling — by the power of suggestion.
While we were working on the piece, I saw the movie Annihilation, and dreamt later that night that I was falling down a deep sea crevasse, dodging strange fish and spear-wielding underwater creatures the whole way down. I batted away tentacles reaching out from the sides of the crevasse, only to discover upon opening my eyes that I was batting my wife’s hands away as she tried to wake me up from my nightmare.
This raises the simple, but powerful idea that a felt experience can be interpreted by the mind as a totally different one. We’re all experienced it: Phantom cell phone vibrations, that invisible bug you felt crawling up your arm, that cold dry bench you momentarily feared was wet. We feel something — a touch, a rub, a vibration — and our mind up-samples the sensation to give it more fidelity. It gets built into a narrative.
It turns out that the same qualities of vibrations that make something sound wet and squishy can make them feel wet and squishy.
This is exciting for us designers. It creates opportunities that our sense of vision, for example, does not allow for. Sound designers for film are intimately familiar with this phenomenon and make use of it often. The sound of a blender can stand in for the jets of a space cruiser. A wailing bear can become Chewbacca’s voice.
It turns out that the same qualities of vibration that make something sound wet and squishy can make them feel wet and squishy. This reminds us of the profound commonalities between materials and experiences. It allows us to use the universal qualities of vibration and sensation to translate feeling into meaning. This translation is messy, rich, and imperfect. We had a lot of fun playing with it while making Seated Catalog of Feelings.
Sounds that you can feel
Physics tells us that all experience is born of vibrations. Depending on the frequency of these vibrations, we perceive them as light, sound, or touch. There happens to be an overlap (around 600 Hz) between the ranges of audible and tactile vibrations, so that we can both hear and feel them.
This simple fact allows us to create tactile vibrations with the same tools that we use to create audible vibrations (aka sound). This allowed me to approach the design of feelings for Seated Catalog of Feelings in much the same way as a sound designer would work. I used a music editing program to compose the sounds. The difference was: instead of listening to them in headphones, I was feeling them in the seat of my chair as I worked. I was listening with my body.
I somehow knew what to do when the “avalanche of frozen peas” felt too mushy, more like soggy potatoes than hard little frozen peas…
As a sound designer, I have a bag of tricks for making things sound the way I want. I can synthesize, sample, pitch shift, time stretch, granularize, filter, EQ, layer, and sequence different sounds to create a desired sensation. But what is desirable in the case of tactile sensations? How do you design something that feels right? What does it even mean for something to feel right?
The similarities of heard and felt vibrations made the process very intuitive. I somehow knew what to do when the avalanche of frozen peas felt too mushy, more like soggy potatoes than hard little frozen peas: Just shorten the decay and raise the pitch of the synthesizer patch I was using.
In the same way that the repeatable characteristics of the sound of a dog barking — its pitch, its timbre, its rhythm — allow us to identify it as the sound of a barking dog, the parameters of a tactile vibration make it feel like something. It has a shape, a frequency, a cadence that lets us recognize it as driving over a bad pothole, or the face of a tennis racket, or a fart in a hot tub.
As I sat in the vibrating chair and composed, I asked myself questions like: Is it too intense? Is the frequency right? Is the perceptual size of it right? Is the timing and rhythm good? The timbre and texture? Too smooth? Too bumpy? Is the amplitude envelope right? Is the attack sharp enough? The decay long enough?
Take for example the feeling of gargling water. We all know what that feels like in the back of our throat, but what does it feel like to be inside of that throat? What is it feel like to be that throat? The translation of a sensation you would normally feel on just a small surface of your body onto your entire body (while sitting in a vibrating chair) does something amazing with your point of view. It shrinks you. It transforms you. It transports you to impossible places.
An expanded notion of “perspective”
A large portion of the people that experience Seated Catalog of Feelings do so with their eyes closed. This is a beautiful thing about touch: You can still feel things with your eyes closed, and the simple act of closing your eyes pulls your attention inward, expanding the playground of your imagination.
Hearing and seeing are different from feeling. We mostly see and hear things that are apart from us, outside of our bodies. We are clearly separate from the stimuli. Feeling through the sense of touch is different, though. You can feel things inside of yourself and, if they are outside of you, you typically have to be physically touching them to feel them.
Take, for instance, the sensation of a nickel in a vacuum cleaner, you’d either have to be down on the floor hugging the vacuum cleaner, or you’d have to literally be the vacuum cleaner, feeling the nickel bouncing off your mechanical innards.
Like a good book, our sense of touch draws the outlines of a world and lets us color it in as we take it in.
I became giddy when, at the opening of the Senses show, someone asked me “Am I the fly or the shit?” in response to the buzzing sensation of a fly on shit. Someone else asked, “Was I supposed to be Jackon’ Pollack’s canvas?” in response to the splattery vibrations of Jackson Pollack’s canvas.
When you are feeling something that is not alive, or something of a scale you cannot imagine yourself physically encountering, how do you form a perspective? Do you shrink? Do you become that thing? Do you inhabit the body of an inanimate object that has sprouted a consciousness and a sense of touch? Who or what is the subject of the experience?
These questions excite me. They bring me back to the ambiguous nature of touch, in this case the ambiguity of perspective offered through tactile experience. When you’re wearing an Oculus Rift and looking out from some virtual character’s eyes, you know where you stand. On the other hand, when your eyes are closed and you’re feeling the world through your sense of touch, who or what are you? The edges of your body blurred, you experience a delightful ambiguity from a tactile point of “view”.
We stumbled onto this idea in the making of Seated Catalog of Feelings. To experience a cricket rubbing its arms together, you have to be a cricket or, even stranger, a cricket’s arm. To make love to a snail on a bicycle seat, you have to either be a snail or a very tiny human. To feel what it is like to get unzipped, you have to be a zipper. To process the experience, you have to shape shift yourself or stand in the shoes of some inanimate object.
Let’s have more fun with our sense of touch
In 2050, when we’re living in the promised land of full-fidelity tactile virtual reality, making virtual love in our full-immersion transcutaneous electrode stimulation body suits, we’ll look back at Seated Catalog of Feelings as a whimsical amusement. For now though, there’s something exciting about the ambiguity of touch. The way that it, like a good book, draws the outlines of a world and lets you color it in as you take it in.
In these disembodied days, where the majority of the experiences we’re tuned into happen from the neck up, I am attracted to things that remind us to listen to the world with our bodies. As designers we have everything we need right now — most of all an underserved sense of touch — to craft new and surprising experiences for our bodies.
Eric Gunther is Creative Director and Cofounder of Sosolimited, where he creates sensory spaces driven by design and technology. He started working with vibrations during his Masters thesis at the MIT Media Lab, and has been creating artworks for the sense of touch ever since.
