The no-screen experiment

What happens if you make an app without the screen?

As I finish up my Computer Science degree at Reykjavik University, I’m making a musical iPad app. Since it’s my first proper iOS app and thoughts are flying all over the place, I’m in sore need of a vantage point. So why not write about it?

iPads are magical friggin’ creatures. Just look at them. They’re sitting there in your hands, flat and silent. And then, once you click a button, their LCD mouths open up to spew visual koans for you to interact with. It’s marvellous.

But iPads don’t just have LCD mouths. Crucially, they also have eyes and ears. They have a physical shape. They have a sense of balance. They know if their screen is being touched in a million different ways. They can even tell if it’s bright or dark outside of its immaculately sculpted metal enclosure.

Wouldn’t it be interesting to use some of these other senses? Like, really use them? Sometimes you have to turn down the volume of one thing to let the other things reveal themselves. So how about diminishing the role of the screen to let the other elements shine?

This is what I’m aiming to do — I’m going to turn this creature on its head and try and keep its LCD mouth shut as much as I can. Instead, I’m going to use gestures and audio to allow the iPad and the user to communicate.

But why ignore the screen? iPads are nothing but a big, gorgeous screen! Well, here’s a motivating example: If you’re watching a show and see your musician burying his head into a screen, you’re going to feel a bit left out of the party. As a laptop-toting musician myself, I try very hard to keep all screens out of sight — my own sight included. I’ve found it on my own skin that it enhances rapport between the performers and the audience, not to mention between musicians on stage.

And as to iPads being nothing but screen? Well, on top of that screen is a gestural interface spanning almost the entire device. So if you squint your eyes hard enough, you might just as well realize that iPads are nothing but huge gestural interfaces with a screen underneath.

There are a million ideas that come to light by adopting this viewpoint, but here’s the path I’m taking: A looping app that can loop-e-de-loop by using your fingers and not watch that glowing screen! It’s designed to make your iPad into just another little instrument you plug into your setup. No laptop needed to have fun.

The feature list is small, but should be enough to keep the user engrossed. It has two looping slots with the usual rec/play/overdub features, reverse mode and an octave shift mode. It will not have tempo sync. You will control all of these features with just your fingers. No looking at the screen will be necessary — you’ll know what you did by listening to the result of your actions.

Of course, there will be a nominal visual UI (otherwise the app would be too obtuse and Apple probably wouldn’t even let it into the App Store), but I’m going to try and let this guiding principle steer me.

The app already has some basic scaffolding, I’ve done some prototype user testing and I’m really excited to see where it goes. I’m doing it as a final project for my B.Sc degree in Computer Science at Reykjavík University, under the firm hand of Marta Kristín Lárusdóttir at the CRESS Research Centre. What we are looking at is the user testing aspect of creating an app like this.

This project is turning out to be incredibly fun — way too much fun. There are tons of things to think and write about, so hopefully I can post another one of these missives soon. Until then, I’ll be in my shed.

photo credit: Raúl Hernández González and Matthew McVickar

    Eiríkur Orri Ólafsson

    Written by

    Trumpeteer, noise maker, iOS geek. Iceland.