Prototyping…and more prototyping
The past week we saw a whirlwind of progress in our mission to beat ugly point-and-click text input in virtual reality. But before we talk about that, since this is our first Medium post on progress, I’ll just update our millions of fans out there on our idea so far.
Scribble
The inspiration for our idea came from Nishta’s (one of our team member’s) Apple Watch. Whenever she got a text, she was able to do quick replies from that tiny Apple Watch screen by literally tracing out the letters she wanted to send. Because computer vision tech is so good these days, Scribble, as Apple calls this tech, is pretty darn accurate. And especially for short replies like “K” or “Thx,” Scribble can be pretty fast and comfortable when you don’t want to pull out your phone.
Wait a minute…Fast, comfortable, an immersive experience that involves tracing out letters. This seems to solve all the problems of ugly point-and-click. We thought, what if we could bring this fascinating, immersive handwriting experience to virtual reality?
Rapid Prototyping
So back to last week. Our rapid prototyping was all about figuring out how this Scribble tech would look in virtual reality. Would people scribble by waving their arm around in the air or scribble on a surface like an iPad or a smartphone? Is Scribble really faster, really more comfortable than ugly point-and-click? You can check our complete presentation with some data here, but the short answer is…
They’d rather not wave their arm around in the air. And yes, Scribble really is faster and more comfortable.
Scribbling our way forward
Post-rapid prototyping there are really two big hurdles we have to cross for Scribble. The first is getting feedback from a mass audience of people. We learned a lot from rapid prototyping. Seriously. Like the fact that people can come up with sooo many different ways to trace out letters on a phone from holding it in portrait mode, using a stylus, and writing out letters one at a time to holding it horizontally and writing out entire sentences. But there’s still a lot to learn from a mass audience of users. In what scenarios would you use Scribble over voice or point-and-click or even the Logitech VR keyboard? Would it be for Youtube commenting or password entering or maybe brainstorming on a virtual whiteboard? (If you have an idea for Scribble comment below!)
But to get some more feedback from the virtual reality enthusiasts out there, we’ll be asking questions and possibly using ads on targeted websites like subreddits or Oculus forums.
The other hurdle is just getting a functional prototype of Scribble working. Luckily, we discovered that Google has a super powerful handwriting input reader that runs on a distributed neural network and is probably 1000 times better than any handwriting recognition model we can churn out in the next few weeks. Even more luckily, somebody built out a pretty neat API (not an official API though) that pings Google’s models with handwriting input. So, two of us will be a) setting up a React Native app that uses their recognition model and returns computer characters and then b) sends those computer characters into a Unity virtual reality app. Stay tuned for how our prototype turns out!
Ta-ta for now!
Tariq
