(Not) Redesigning the Virtual Keyboard.
Solving design problems in virtual reality will require a different method of thinking than what most of us UX designers are used to.
Take our old friend the QWERTY keyboard. Currently, most VR users aren’t sitting down at their desks and doing office work, but as with most technologies, VR will normalize and work its way into offices and workspaces everywhere. These new workspaces will be designed to be less reliant on traditional 2D interfaces and replaced with more natural gestures and physical inputs, but we still need ways to type.
There are already a lot of VR keyboards, but no current design is quite efficient enough for long-term comfortable use in every-day use cases.
Current available typing solutions.
Leap Motion
The classic solution is the simple floating keyboard UI.
This design is most familiar to users as it’s the closest to your typical keyboard. The downside is that the lack of a tactile response or surface for your fingers to hit makes accurate and consistent typing very difficult.
NEC Armboard
Another interesting design is NEC’s virtual keyboard for your arm.
In this demo a keyboard interface is programmatically pinned to the user’s arm and is great for easy access. Unfortunately, the user is limited to only being able to type with one hand and is also unable to operate another controller in tandem with the keyboard. This input method may work in restricted environments but is not comfortable or flexible enough for an efficient workflow.
I’ve given it a shot…
When I first started tackling this problem of keyboards in VR I tested out grouping keys to fingers in the same range that we currently use with our physical keyboards. Those keys could be pressed by simply raising or curling your finger, depending on the the orientation of your hand, to the designated UI space.
I quickly realized that this doesn’t solve any of the core problems with virtual keyboards. Not only do we still have the problem of tiring arm positions that could cause strain (tendinitis is not fun), but it would be difficult to learn how to type quickly and the probability of hitting the wrong key is high. Good luck reaching up and hitting any keys mapped to that inflexible ring finger of yours.
This year at Google I/O the Daydream Labs demonstrated their drum keys.
This keyboard is operated by using tracked controllers with digital drum sticks sticking out of the ends of the controllers and bopping on keys like a game of Whack-A-Mole.
It’s a fun and creative solution and appears to work well for quick interactions that you may see in a game but isn’t really a good solution for long-term use. Not only does the likeliness of error seem high but if you type all day you’ll end up with some pretty tired arms. One benefit to that is after extended use you’ll be able to defeat anybody in an arm wrestling match.
How can we make this easier?
Turns out, the simplest solution is right in front of us: our actual keyboards. Using the same technology that the Vive uses to track and display its controllers, we can track our real-world keyboard and put a virtual keyboard representation of it within our environment.
The user will see the keyboard in their virtual space and can type on it directly in their physical space.
Yeah, it can really be that simple.
This works because we solve all of our problems in one fell swoop. It’s familiar, there’s no barrier for adoption, and no new training is required. It provides an anchor to the real world and doesn’t require the pinning of intrusive UI in our face. We could even make and mass produce this today because the technology is already here.
This opens up a whole world of fun possibilities to change our keyboards from every-day objects to fun and engaging tools in our environment. Key maps could appear based off of the active program window. Video games could take control and re-model your keyboard to match a control panel or the game environment. Keys can programmatically light up or animate based on presses or react to the environment around you.
The possibilities for customization are virtually endless.
So there we have it — a difficult problem solved by an almost-too-easy solution that also leaves room for fun modifications.
Just because something is virtual doesn’t mean it has to be artificial. Real-world objects can be utilized to make the environments work in conjunction with one other. Some things in the real world already work well and the keyboard is just one of those things.
Hi 👋 I’m Jared . I’m a UX designer in San Francisco, design community advocate, resource provider, and knowledge sharer. I’ve been designing various VR projects from games and experiences, to operating systems and education tools. Let’s grab coffee.
