Remote Content Usability Testing for LifeSG

For the past few months, I’ve been working on LifeSG (previously known as Moments of Life). As the UX Writer for LifeSG, my role is to research, write and test government content for:

• Support for your job search
• Retrenchment benefits and measures

At first, we planned to meet citizens face-to-face for usability testing (UT) sessions in May. But COVID-19 happened and remote working became the norm. We quickly got our heads around and ran our first round of remote usability testing for content from 13 to 24 April 2020, with 10 participants.

What is content usability testing?

We’re all familiar with usability testing for design flows and interaction patterns. With usability testing, we collect insights, findings and observations about how people use a product or a service.

But what about testing the content in a product?

Content usability testing focuses on how users interpret or interact with the words. It asks the user questions on:

• Clarity: Was the content clear and easy to understand?
• Usefulness: Did you find the content useful or relevant?
• Voice & tone: What is your impression of the voice and tone?
• Findability: Are you able to find the information you need easily?
• Confidence: Are you confident in knowing what to do after reading this?

Quick methods for content testing

There are a variety of content testing methods out there. Depending on your project needs, objectives, resources and timeline, use the methods that suit your team best. But definitely test your content, since users rely on words to use or navigate your product.

For LifeSG’s content UT, we used 3 methods: highlighting test, comprehension survey and readability survey.

1. Highlighting test

First, we asked the participant to read the content at their own pace. They are encouraged to think aloud and share their thoughts at any time they wish to.

We then asked the participant to highlight the information that they found useful, clear or relevant in green, and the information that they found not useful, unclear or irrelevant in red.

2. Comprehension survey

Remember comprehension multiple-choice questions (MCQs) back in school? Comprehension surveys are simply questions to test whether the participant understands a piece of content.

After participants highlight their responses, we invite them to answer a series of simple and factual questions about the content they have just read.

Participants are free to refer to the content while completing the survey. We want to test their understanding and recall, but not their memory.

3. Readability survey

Taking inspiration from the System Usability Scale(SUS), our team needed a quick way to measure the success of our content. We came up with 4 statements and asked participants to give a score based on:

• ease of reading
• clarity
• usefulness
• user confidence

The scoring interpretation is kept simple and we looked at the average of participants’ scores for each statement. While we wouldn’t dare to claim statistical significance, this at least gave us a quick idea of how well our content did.

Tips for remote usability testing

With all these COVID-19 measures, the only way to properly and safely test our content is to do so remotely. In this section, I will share the lessons learnt and the tools we used for moderated, remote usability testing.

Zoom

For video meetings and screen sharing with participants.

Google Docs

For participants to highlight, comment and give feedback on the content we’re testing.

OBS Studio & YouTube Private Streaming

For project teammates and stakeholders who want to sit-in for usability testing sessions.

Usually, we would use the observation room in our office’s UT lab. But that option’s unavailable due to COVID-19… so how could we make room for UT observations online? To recreate the observation room’s one-way mirror environment, I used OBS Studio to stream the Zoom sessions to a private live streaming link on YouTube. We meet the participant for UT on Zoom, while observers watch the session on a private live stream.

The streaming links are unlisted, private and strictly by invite only. We also make sure to inform and get the participant’s agreement.

Get comfortable with silence

For remote content testing, expect long stretches of silence while the participant focuses on reading the content page to page. Get comfortable with silence, be patient, and avoid fidgeting or interrupting their concentration.

Sometimes, participants may instead feel awkward with silence and wonder if they’re being helpful. With more practice, you will start to know when is the right time to give proper feedback and assurance. You can even encourage them to think aloud and share their thoughts as they read.

Test your content… even as you work from home!

Content testing is not expensive but it requires time and effort. As a result, it’s common for project teams to put content testing aside as a ‘nice to have’.

Yet, words play a critical part in how people understand, learn or use a product. We might not have time or budget to test every single piece of content, but we can start small by replacing lorem ipsum with actual instructions or including content questions in usability testing.

At GovTech, I’ve been lucky that the 2 project teams I’ve worked with saw the importance of testing content. By paying attention to how people read, interpret, and respond to content, we can know how to better communicate, structure, and design information.

Many thanks to:

• Michelle + Fugene for their guidance on usability testing
• Selig for note-taking and the funny gifs + baby photos during waiting time.
• Tobi, for the time spent recruiting participants and hard work liaising with agencies

Lastly, I’m super grateful for a supportive LifeSG team, where everyone has put in hard work and long hours to constantly improve this product.