Just say it

I think we are about to see a shift in human-computer interactions. Before the Augmented or Mix Reality will grow to its full potential a much more simple and straightforward, but profound change will take place.

We are going to mumble to our computers. A lot.

Productive tablet

iPad Pro is probably the most underutilized computing platform. Touch + Pencil together are perfect for fast, intimate, and precise input. What is missing?

Keyboard shortcuts. Not the keyboards though, just shortcuts.

If you are a pro user what you want is fast interaction. You want to be effective in changing a mode, adding an object or accessing a property. On a desktop computer, the way to do this is the keyboard.

A pointer is there to provide context (selection, hover) and the keyboard is there to tell a computer what to do. Mostly with shortcuts like Cmnd + a key or a short text search + Enter.

Point and say

When the touch revolutionized the way we interact with computers it also took away the current context. It was okay for a small mobile device. But without the context of a pointer and ability to say what to do with something you are pointing at, interactions on tablets are cumbersome.

No, I do not want to tap and hold to copy a rectangle. I want to point at it and say “Duplicate”. On a Mac, one can do that via the cursor and Cmnd+D.

Touch and say

I guess you see where I am going with this. I think user should be able to just touch and say. Tap on anything and give a comand in a calm voice.

No need for another input device. Just tap and say what you need it to do. The computer already has all the context and all the capabilities necessary to understand. You are close, no need to raise your voice. Say it casualy. Say it softly.

Just tap on a text segment and say “cursive” or “align right”. Tap on a canvas and say “add artboard,” raise your watch and say “skip” (say while wearing AirPods and listening to music), look at notification and say “respond”…

Look and say

With the FaceID (and MLKit) more than ever Apple is ready to enable us to interact with our computers in the most personal way: by telling them what to do with what we are looking at.

So just say “launch Mail” looking on the right side of your iPad to open the Mail app on a side from your current app. Dismiss or respond to notifications with just a look and a short command. Look at the input field and dictate the text you want your computer to put in it.

The most personal way to interact with your computer is coming. And it is going to deliver what is missing in touch-first devices: the fast contextual interactions.

Enabled with the voice, big screen tablets will redefine what the desktop for a creative professional is. There will be a lot more mumbling and a lot more will be done faster.

Empowered with the voice-enabled shortcuts humans will be prepared for the next big change when the screens will be unbundled to surround us and augment our world.

I got a great feedback on this article and flowed up tying to cover questions about the speed, the social factors and the noise. Check out The case for the voice interactions.