Helping non-verbal people find their words

Dean Birkett
Jun 20, 2020 · 7 min read

AssistiveWare develops assistive technology software for iOS and Mac OS, creating applications for people with physical, vision, communication and reading impairments.

Proloquo2Go is their flagship application, a symbol-based communication app for children and adults who cannot speak. As the UX lead for the project, we needed to introduce a powerful search function to help professionals, family members, and the communicators to find their words. I performed research, planned and facilitated the ideation workshops, designed, prototyped, and tested the solution through multiple iterations.

Visit Proloquo2Go

Proloquo2Go search in progress on the iPad, with 16 buttons showing which match the results, and a keyboard at the bottom.
Proloquo2Go search filtering

Client: AssistiveWare
Platform: iOS
Role: UX Lead
Date: 2017

Awards: Aquent agency 2018 UX Talent award winner

The Objective

Proloquo2Go is a complex app, which is used by non-verbal children and adults with various diagnoses or acquired conditions. Enabling these people to communicate is critical.

In the Oxford Dictionary at present, there are over 171,000 words. To find a word in a paper copy, you would go through the alphabetised list until you reach the word that you are looking for.

Proloquo2Go has 20,000 symbols that are used by the communicator to express their thoughts, or simply chat. These symbols are placed in categories, so for instance, the word ‘beach’ may be found in the category ‘Places’, and in the subcategory, ‘Outdoor places’. It can also be found in ‘Things / Nature’.

There was no way to search for words in the app, so it was often like looking for a needle in the haystack. A PDF document was created to assist the user, but of course, this was not ideal.

AssistiveWare needed an in-app solution to allow the user to find their words quickly and easily.

Two pages from the Proloquo2Go PDF highlighting where in the app to find words.
Examples from the PDF document

The app was due for its fourth major release, and this fourth release included two new features (Search, and something called Progressive Language) and a redesigned Backup area, which allows the user to backup to a number of cloud services.

The previous way to access operations in the app was not scalable, so we needed to find a way to include these features, and make them easily accessible.

Overlay showing ways to access different views in the Proloquo2Go app.
Previous solution to access operations in Proloquo2Go.

I started to ideate possible ways to include the new features and to ensure that it was a more scalable solution should something need adding in the future.

The way that you currently accessed the ‘Grid view’, ‘Recents’ and ‘Typing view’ was through a ‘Views’ button in the lower left. My assumption was to re-use this area so that those familiar with these operations would assume this to be the area where they would find the new features.

The icon in use here was one that is usually used to imply copying documents, so I also wanted to revisit that.

Early sketches involved a menu bar that would slide up, but this would be an issue for people using keyguards. So I wanted to look at ways to use the grid to include the new features, that way those who did use keyguards did not have to purchase a new one because of the design tweaks.

A keyguard over the top of an iPad, with Proloquo2Go on screen.
A keyguard over the top of an iPad, with Proloquo2Go on screen.
A Keyguard being used with Proloquo2Go.

I created a concept using a new icon to access the ‘Tools popup’ and placed all of the functions needed for this area into the grid. The new icon was then tested on members of Proloquo2Go’s Facebook group to see what people expected to happen after tapping it.

“Looking at the screen now, I see there is an extra button in the lower left, so I would go to that new symbol that I haven’t seen before”.

These results were positive, and this was communicated to the development team.

An annotated screen specifying the requirements to the development team.
An annotated screen specifying the requirements to the development team.
Annotated screenshot which was shared with the development team.

Alongside finding a more scalable solution to access the new functions, we also naturally needed to define ‘Search’.

Who was Search for?
How did they expect it to work?
What are the clinical best practices?

The Process

I started the project by creating a UX Questionnaire and asking the Product Team to complete. The questionnaire was used to establish a number of things, such as the following:

  • Who is in the core team for this project?
  • Who needs to approve things before proceeding?
  • What are the measurable business goals?
  • How do we know if we have been successful with this project?
  • Who is the target audience?
  • What do we know about them?
  • What do they want to achieve?
  • Who are the competitors?
  • Are there any key dates or milestones?

This document is a grounding, which helps to establish what we know and equally important, what we don’t know.

The core team was established. Myself as UX Lead, the Product Owner, Project Manager, and a Clinical specialist made up the rest of the team.

The kickoff meeting

Prior to the kickoff meeting, I performed a competitor analysis, did some desk research, and naturally prepared for the kickoff.

I don’t like kickoff meetings to be merely a presentation followed by a discussion. I like to be more hands-on, so with that in mind, I needed to establish what I wanted to get out of this meeting.

My ideal outcome was to have a brainstorming session where we defined who would use this feature and to sketch some initial ideas out to test with these users.

With this in mind, the structure and goals were:

  • Share research findings
  • Define the problem we are solving today
  • Define the audience
  • Pose a problem to solve
  • Sketch solutions
  • Present our ideas
  • Critique the suggestions
  • Collaborate on a possible solution, or solutions
An elevator pitch asking the audience, opportunity and solution, and a task for the sketching workshop.
An elevator pitch and one of the tasks set in the workshop

Primarily ‘Search’ was to be used by teachers, parents, caregivers, and speech-language pathologists, and finally the communicators themselves needed to have access should they have the ability to use this feature.

Three problems were posed to cover a variety of use cases:

  • How would you find the word ‘jigsaw’? It could be a fun puzzle or a type of saw.
  • How would you find the word ‘sweetcorn’? It could be a food item or it could be saved in storage.
  • How would a bilingual user find a Spanish word when they are currently using the English vocabulary?

And the team were tasked with sketching possible ways to solve these problems.

A low fidelity sketch from the workshop showing how someone might find the word jigsaw in the app.
A low fidelity sketch from the workshop showing how someone might find the word jigsaw in the app.
One of the sketched solutions.

Afterwards, each team member had to present their solution, and the remainder of the team critiqued their design. What works well? What needs refining? What technical issues may there be? What may be confusing?

After a round of sketching and critiquing the team felt that one particular route seemed to be the most appealing. It was a route sketched out by a number of team members, so internally we felt it was a good one to test.

Three whiteboard sketches showing the process of searching for a word.
Whiteboard sketches showing the process.

Validating the idea

Sketches can work with some audiences but to ensure they are not misinterpreted I felt it was wise to increase the fidelity for this particular audience.

Having already created a Proloquo2Go template file it was easy to dive into creating a high fidelity prototype.

Sketch app showing various screens from low to high fidelity
Sketch file showing the transition from low fidelity to high fidelity.

I created assets around the three tasks I created, which I wanted to test out on the participants.

Task one:

Your child loves collecting stamps from all around the world, and you’d like to help them to share their interest with others.

You have started your sentence and opened up the “Things” folder to see if you can find the word ‘stamp’.

Unfortunately, you are unable to locate the word ‘stamp’, so you would like to search for it.

How would you do this?

Task two:

You would like to write the sentence, “Play is fun”, but you can’t recall where the word ‘play’ is located in the vocabulary. You remember searching for this word recently using the new search function.

Use the search function again to try and locate the word ‘Play’

Task three:

Your student, Mateo is bilingual, and speak both English and Spanish at school. At home, he only uses Proloquo2Go to speak Spanish as his parents do not understand English very well.

You would like to model ‘carpenter’, which is Mateo’s fathers’ profession, but would like to search for this in Spanish, so Mateo can communicate this at home.

Search for the word ‘carpintero’ to find how to model this.

With the prototype created it was time to source participants to help us find out what works and what doesn’t.

Thankfully, finding willing participants has always been a relatively easy task at AssistiveWare. The apps are life-changing to many, and the opportunity to help us make them better never went amiss.

After testing with a number of users and making some iterations of the design, we felt confident that the route proposed was the direction we should be going in.

“I found it very easy. You guys have thought about everything. I think people will be thrilled that this feature is coming out, and it’s so user-friendly. The interface is clean. I think it’s great!”.

The results

The results were hugely positive, it was described by some professionals as being a ‘game changer’. Sales improved, but in this instance, it is the qualitative feedback that makes everything worthwhile. Hearing first hand about the success their child is having is the most important thing.

Two Facebook posts showing positive feedback from the users of the app.

Dean Birkett Design

Removing barriers to include more people, with design solutions that benefit all.

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store