I Designed a Voice App that Helps You Prepare for the US Citizenship Test.
and any other test you can think of.
Getting more out of a commute, workout, or a chore has been the goal of many people.
That “job-to-be-done” is intense enough, frequent enough and common enough to essentially build up whole new industries like podcasting & audiobooks. Audiobooks and podcasts would not be popular without the mundane.
The Opportunity
In the 2018 remake of the movie Overboard, a single mom has to juggle the responsibilities of two children, multiple jobs, and a full-time nursing school course load.
She still needs to make sure meals are cooked, children are fed and clients are happy all while preparing for her examinations to become a registered nurse. One scene that stood out to me on Overboard was when our protagonist, played by Anna Ferris, is delivering pizzas and listening to test prep CDs for her nursing examinations. The CD would read out a question she would have to answer it before the CD moved on to the next question.
This interaction is one-sided. What if the person doesn’t know the answer? They would have to rewind the tape or CD and risk crashing if they wanted to noodle on a question for a while longer.
So far, highly technical or conceptual content is too hard to digest in audio format. That is because our working memory as humans can only retain up to three items at a time.
In another article I wrote called, Yes, Audio Book UX Design is a Thing I ask similar questions:
Technology is changing, and the new ways you can listen to an audiobook are being invented every day, innovation waits for no one, especially for big publishing. By not doing anything, these big publishers are ripe for disruption. Allow me to present a few problems Audiobooks do not currently solve:
• What about textbooks? You know, the things that take up a majority of a students time in college? How can you navigate to different topics and review for a test while you do chores around the house or while working in that part-time warehouse job?
• What about non-fiction, skill-intensive book? Most of the ones I have can be so dry that they are impossible to listen to, or while driving you miss an opportunity to marinate on a principle. Here’s a screenshot of a very principally dense audiobook on audible. Ray Dalio’s Principles.
My conclusion is this: audiobooks may be good for passive learning, but when active learning is required, which is the single focused deep work and practice it takes to learn a new skill or acquire new knowledge, isn’t really supported in a linear audiobook format.
What affordances are there for a commuter? None. The bookmark feature can’t be used while driving unless you take the risk of losing focus on the road.
I used to think that we just had to write better audiobooks to solve this problem. Which is true if you can do if you can thread a complicated concept into a compelling narrative like in the Go-Giver Leader or The Omnivores Dilemma. But you still cannot address a whole curriculum in an audiobook for things like the GMAT, LSAT, MCAT tests.
I’m convinced that you cannot effectively replace good ol’ fashioned studying in the audio format.
Until now.
How Conversational Design Can Solve This Problem
We now have the technology to tackle heavy curriculums. With new voice assistant enabled apps for platforms such as CarPlay, we can effectively study for complicated examinations while commuting or doing chores.
I think about big examinations or interviews I have done well in, and there was my dad, mom or tutor who was reading flashcards with me or walking me through a concept.
A service that does this well and effectively replaces the need to use someone else to help you study is Quizlet. This website makes it easy to learn new concepts because of it’s gamification of learning.
So instead of reinventing the wheel, what would a Quizlet voice assistant look like?
What Quizlet Does
Quizlet has a bunch of activities that allow you to conceptualize and cement understanding in a topic. While auditing the functionality of these apps, I decided that three of them really worked in a voice only interface:
Learn is the flagship gamification feature of Quizlet, it creates a safe environment to fail and see your progress in understanding a concept or vocabulary for a test.
Flashcards are classic and can be done in a conversational setting already. Think about your tutor reading them off to you while you answer them by just using your voice.
Spelling the word out doesn’t necessarily help with comprehension of a topic, but it does help with retention of vocabulary which will give you the building blocks you need to better answer flashcards and tests.
System Persona
In the spirit of me getting nostalgic about my days of being tutored in grade school, I think the best approach for designing the system persona is a tutor figure. Which we will call Quizlet Tutor.
A great tutor guides and coaches the student. So the persona need’s to communicate patience and afford responses that helps the user feel like they are in a safe place to learn.
If anything this should allow the user to shed in inhibitions because there are no social pressures to be cool or look dumb while you are alone in your car. If done, right, the user should shed their shells and facades to give themselves to the process.
Use Cases
The following use cases need to be considered while designing the app, such as the following list. I bolded two uses cases I will focus on for this case study.
- User successfully studies five facts using the flashcards feature.
- User successfully learns five new Spanish words with spell feature.
- The user has successfully walked through a concept using the learn feature.
- User successfully prepares for a math test using learn or flashcards.
- User successfully prepares for a US citizenship test using the learn feature.
- User successfully prepares for a certification during the commute.
Customer Persona
I married into a Latino family. And just recently my in-laws just passed their citizenship test.
The US Citizenship exam is an oral examination that has the person being tested recite facts about America.
Unfortunately, the test is very arbitrary. If you have a thick accent and poor English, the test can be quite rigorous.
My wife, who grew up in the USA since she was two, on her 18th birthday only had to answer two softball questions. She has an American accent and has been through the US school system.
The thing is this, you have no idea how hard it will be for you, so taking the time to prepare is well worth the investment.
The difficult thing is that many immigrants work multiple low paying jobs or have too many obligations to do it well. So instead, they put off doing the test until a new political climate creates a sense of urgency to do so. That is what finally got my in-laws to go through the process after being residents for 20+ years.
So we have to assume that a person that would use this voice app would be like our heroine Anna Ferris in Overboard, studying for a big exam in her car.
So I am going to focus on a car play app that fulfills this need.
Scriptwriting
When designing the scripts, it is always good to verbally hash out a conversational flow to get a feel of what sounds natural. The challenging thing about conversational interfaces is that people have been using voice long before computers have existed. They do not need to be taught how to talk.
So if an interface sounds awkward, it will prevent the user from fully cooperating with it.
So to combat this, I sat with my wife in the car while she was driving and I quizzed her us citizenship questions. Something I learned while ‘tutoring’ my wife was that if there is a lapse in memory and the user forgets a fact, you don’t say “I’m sorry, that is incorrect”.
That statement may seem harmless, but it does come off as impatient, like a coach who points out every mistake of their players.
In order to make the learning environment safe to fail, we need to design recovery phrases that make the user feel comfortable with failing.
Script one — Flashcard Happy Path
Before exploring edge cases, you should first address the best case scenario. It is looking at the perfect scenario helps you see what could go wrong in the flow.
Script Two — Learn Happy Path
The cool thing about Quizlet is that it uses technology to help you prioritize your studies. It will give you topics to work on based off how many you mastered. So all the user should need to do is invoke Quizlet and the persona should serve up relevant next steps for the user to study.
Instead of giving an overview of all available things to study, Quizlet Tutor will suggest topics that the user hasn’t mastered yet. In the learn feature, there will be pre-made audio portions to go over topics with Quizlet going over key points with the user between each recording.
Script Three —Spelling With a Recovery Option
What happens if the person doesn’t know how to spell something? In this script, I attempt to recover after someone has misspelled a term.
If the spelling isn’t exact, the tutor will give the option to try again or get the correct answer spelled out for them. Either way, the persona should make the user feel comfortable with failure.
Flow Chart Exploration
Using Lucid Chart, I can start to add edge cases and alternate paths later in the design process.
While testing the flow, I decided to make sure the system persona, the tutor, would insist on the user trying again instead of asking if they wanted to try again.
So instead of saying “do you want to try again?”, Quizlet tutor will just ask, “try it again.”
If the user doesn’t answer a prompt right away, a alternate soft ball question will re-prompt the user in 6 seconds.
Testing Plan
I mocked up a flash card test using SaySpring as my prototyping tool.
I will usability test with 5 people to see if the script works well. I am only going to pass them through a happy path scenario so I can flesh out edge cases and friction points in the script.
I will set up the test by telling the interviewees with the following prompt:
You are an immigrant in the United States and you are preparing for a citizenship test. You currently work full time and have side jobs to make end’s meet. A friend of yours suggested this voice app called Quizlet Tutor that allows you to study while performing other tasks.
Let’s say that you are driving your car to work and you want to study some terms via your flashcards to prepare for the US citizenship test. How would you do that?
I then will observe how they converse with the Quizlet Tutor and look for things I can improve in the design. If the prototype failed, I’d adjust into a wizard of Oz style role play where I took on the role of the tutor.
You can test the prototype here.
Testing Insights
User 1
User 1 went through to the flashcards and got stuck on question 1. She instinctively told Alexa, “I don’t know”. I asked her what she expected to happen if she said “I don’t know” and she said she expected Alexa to to give her a hint or the answer.
Iteration Note: After this test I added hints into each question just in case they said “I don’t know” or any variation of that term.
User 2
User two was able to navigate to the correct flashcard deck, but her first instinct when she didn’t know something was “I have no idea” and she broke the flow. So now I have another response to put into the prototype.
I finished user 2’s test wizard of oz style and the flow worked pretty well. It is just the edge cases on the possible responses that keep crashing the test.
User 3
User three went through the whole flow smoothly. This is because knew the answers. It seems that the experience is rough if the user doesn’t know the answer. So I need to be more aware of any responses a person might say if they don’t know.
User 4
The user was confused by the pacing in the question, “what do you want to study? Us citizenship test or english vocabulary?” He just said, “yes”. So I need to ad a pause between the deck options.
I also noticed that whenever they said the wrong answer, the prototype would break. Thats because I didn’t have a proper recovery response for if they said the wrong answer. I could insert a prompt that is triggered whenever a wrong answer is given.
For example, Google could say, “Not quite, here is a hint…”
User 5
The user went through the flow just fine because he know the answers, but he had one comment:
“When Google said, ‘you have improved significantly, I thought it was over, so I was a little surprised when it kept going.”
The big take away here is that I need to make affirmations less formal.
What I Have Learned so Far
As always, I understand that the hardest things to design are the recovery phrases in the prototype. To efficiently come up with a proper curriculum, I would need more time to efficiently come up with hints, recovery phrases and different flows to help the user get a handle on content.
What I can design better is a framework for creators of curriculum to follow so they can plug in the content for me.
This is version one of this case study. It will be updated when future findings are made.
