The VoiceBook Saga Part III: Return
Part three of our 20 month quest to reinvent eBooks using voice recognition, in which we almost release an absolute hoot of an app… but don’t due to a unexpected insight.
For the complete context, please read Part I and Part II of this essay first. The tl;dr version is that we had developed a prototype of a new voice recognition powered, semi animated tablet reading product for parents and kids that we called VoiceBook as part of our plan to ‘disrupt eBooks’ and give adults creative superpowers … but we’d failed to ship anything due to technology, creative and business concerns and the project was put on hold.
Part III: Return
January 2017 — July 2017
After making the decision to pause the project in July 2016 the project went on the back burner for the rest of the year… but it didn’t die. The concept was just such a perfect alignment with our company mission that many of us wanted to keep it alive somehow.
Two important things happened to bring it back from the dead.
First, Liam just kept working on it. He spent time around the edges of other work reflecting on what we’d learnt to date about the tech and how to best process voice input and use it to provide triggers for animations.
He built a de-bugging tool that allowed him to visually inspect the performance of different word types and recognition approaches. For example recognising a single sounds (KeyWordSpottingOnlyKey), recognising several sounds (KeyWordSpottingAll) and recognising a fast paced se of sounds (FastSentance).
He built a version of Where The Wild Things Are into this, one of our favourite books to read aloud, and you can see a demo of this below with some of these recognition types being visualised.
This tool led to various breakthroughs in terms of the quality of voice recognition we could confidently deliver and our ability to test it.
The second important thing that happened was that we began working on an ambitious product roadmap for 2017. We’d made a decision, for various complex reasons, not to ship a major new book in 2016 and by the second half of 2016 we were starting to regret this.
So as we began planning for 2017 we wanted to catch up and launch an ambitious portfolio of books that the Story Studio had been developing since late 2015.
As part of the 2017 strategy, alongside lots of new picture books, we wanted to include several innovative products that would push us beyond our existing category of personalised books for kids— and VoiceBook was sitting there just waiting for us, asking to be shipped.
We took a fresh look.
In the light of Liam’s progress on the tech side and the ambition for 2017 we decided, again, that it was such a perfect alignment of company mission and personal interests that we had to dust it off and give it another shot.
So we took the plunge and included VoiceBook in the roadmap for 2017 and gave it another round of proper funding, a release date and the project swung back into action.
Here’s a copy of the presentation we created to make the case for VoiceBook at our internal ‘NPD forum’, the meeting where we make decisions on investment in new products.
The key changes since the last failed attempt were this:
- Liam had improved the tech enough using his debugger that we could see a working user experience. We had done some detailed testing and were at around 96% success rates with our grammar model and had confidence that UX and careful design could help with the last 4%.
- We agreed that the VoiceBook should be a standalone product, with a standalone story, that was totally aligned with the most fun elements of using voice recognition tech — the experience target was that it had to be ‘a hoot’
- We tried to create more business alignment through inclusion on the overall product roadmap and positioning as part of a move ‘beyond books’
- We agreed to fund a focused team (with some stage gates) including a project producer, an artist, an additional developer and plenty of QA support
So we got to work and assembled the team.
First we hired John Lau, an experienced producer with an interest in interactive story telling. Second, we hired a bold, graphic illustrator we’d wanted to work with for some time called Nicolau.
With this new team we got cracking on developing a 100% voice focused story designed to showcase this new medium at its very best.
We ran various creative workshops that bought together the new team with other specialists from Story Studio and developed an intriguing new story.
You can see some story boards below:
The story we created was quite ‘meta’. The central character is a voice, represented as a changing collection of letters that can morph into whole words at key points.
The story is a quest story, whereby the voice gets cut off from its owner (you, the narrator) and has to find it’s way home.
Along the way it tries out being different voices and sounds, but none of them fit properly. Eventually the voice finds it’s way home where it belongs … just in time to read this story.
We went for a very bold art direction that would suit iOS native animation techniques using physics and states. Here’s some images of the colour scenes.
Throughout the story we tried to implement what we thought were the most enjoyable voice interactions, including a special ‘training’ stage at the start to ease the reader into the concept.
We then cracked on building our VoiceBook.
We planned out the whole project but decided to focus on initially delivering a single scene to a very high fidelity to do a final round of consumer research.
Here’s a demo of where we got to:
We then took this out for a round of feedback from consumers and in the business. We took a clickable PDF version of the story to read with kids and parents along with our single scene with the voice interactivity.
The results were … challenging.
There was lots of little pieces of feedback, but there was one overriding concern —The kids and families were enjoying the PDF version as much as, and often more than, the voice recognition version!
What!?
The voice recognition was working well, but now that it was working it was just … fading into the background. It wasn’t enhancing the story or making the voice ‘acting’ done by the adults much better than they were doing with the PDF version… meanwhile, the PDF version delivered the story, the pictures, the words just as well. And it was these basic elements of the product that everyone was enjoying!
This wasn’t the plan! What was going on?
We got together for an emergency review to understand the feedback and consolidate our own thoughts.
If the voice recognition wasn’t enhancing the story, what was the point?
We weren’t ready to give up and decided to do another two week spike to try and make the voice recognition ‘work’ harder by making the story more interactive.
We rebuilt the first scene to have a less linear pattern and include some loops and fail states — if you didn’t shout LOUD enough or loooooooong enough you wouldn’t move forward.
This made things a little better … but not much.
There was still an overriding user experience issue. The static PDF version of the Voice Book was totally delivering on our brief of making a book that was ‘a hoot’ to read and made adults more confident reading aloud. The expensive and complex voice recognition tech was simply not adding enough.
Our dream of a new technology platform for story telling suddenly felt very hubristic.
Our story really helped adults push themselves and have fun with their voices. But the simple truth was that it didn’t need the voice recognition technology.
The project was already over budget and behind on time, so in early August 2017 we made another, currently final, decision to stop work on the Voice Book as originally imagined.
Look out for the final short chapter, Aftermath, next week, where I’ll sum up what we learned. I’ll also provide links to documentation if you’d like to take a swing at building a voice recognition powered story telling app yourself.
Also, if you like this kind of content, why not sign up for our Strange Tales newsletter to get invites to the monthly events we do exploring ‘unconventional story telling’ at Wonderbly.
