Making digital services more accessible for all — Part 3
Part 3 of 4 of this series explains how testing at scale is needed to discover issues that would otherwise be hidden. If you haven’t read Part 2, you can read it here.
Launching to learn
With agile development, the design of a digital product is never really completed. Rather, we are continuously building, testing, iterating and experimenting. After multiple rounds of testing with citizens in ‘closed’ research studies, the team felt that it was time to ‘go live’, and onboard a large number of testers to inform the next stage of SingPass Mobile’s development.
The first phase of ‘going live’ was to invite government officers to sign up as beta testers for SingPass Mobile. Since it was the first time the majority of them had heard about this initiative, we also had the chance to test our product marketing, how-to-use videos and FAQs. Over 7,000 public officers registered for the beta trial, giving us a deep pool of potential testers to help make the app even better.
Learning from how other GovTech product teams (such as Parking.sg) implemented their beta testing, we chose to onboard beta testers in multiple, small waves, giving us the bandwidth to engage and reach out to individual testers for their feedback. This also ensured that identified bugs were fixed and a new beta release was ready for each subsequent wave of testers, so they would not experience and log the same known problems.
Some issues only reveal themselves when you ‘go live’
The biggest scaling challenge that emerged was how to deliver a great user experience across hundreds of variations of smartphone models, operating system versions and browsers versions, without compromising on system security. We realised that many of the technical issues that emerged were specific to certain models of phone or software where users had not updated for months, or even years.
It was hard to understate the scale of the challenge to resolve such compatibility issues across various devices and software. For instance, even the 30ᵗʰ most used mobile OS version to access digital government services had over 90,000 users. Due to the sheer number of potential combinations involved, the team needed to be pragmatic and prioritise each compatibility issue based on its severity and the number of users potentially affected by using data from technical logs and user feedback.
Another scaling challenge was how to reach out to less ‘digital-able’ users, as they could potentially benefit from SingPass Mobile and biometric authentication that removed the need to remember usernames and long passwords. To address this, we collaborated with PA’s Grassroots Leaders and IMDA’s Silver Infocomm community of trainers to recruit ambassadors with a passion for technology. We equipped them with the necessary knowledge and skills to help build digital literacy among seniors and other less digital-able groups in their communities. Our briefings and training sessions with grassroots organisations gave us additional feedback on how to simplify our marketing and messaging, tailor our outreach efforts to different audiences, and train helpdesk staff to address incompatibility issues and troubleshoot other user-specific problems.
After addressing these challenges, the team felt ready to launch the beta product to the general public and test it on a much bigger scale to garner feedback.
Some problems only reveal themselves when you ‘go big’
The first public release of the SingPass Mobile app went live on the Apple App Store and Google Play store on 22 October 2018. What happened next was not at all what the team had expected.
Error 121 was due to a software error that was not picked up during testing and resulted in issues when a high volume of users tried to register for SingPass Mobile. Onboarding beta testers in small waves ensured we received high quality user feedback and continuously iterate the app. However, this did not provide the installation volumes that were needed to stress test the system.
On hindsight, we needed to run testing scenarios with a much higher number of concurrent user signups than what we imagined for our public launch during internal beta.
Although the problem was quickly patched and resolved, many users had encountered the error and were unable to set up the app. The GovTech product and comms teams reached out to the users who had reported the error through the App store, Play store, social media platforms and government feedback channels to explain what had happened and provide updates on technical fixes.
Given the timely resolution and personalised responses, user feedback quickly turned positive, and adoption rapidly picked up. Four months after its public launch, SingPass Mobile now has over 200,000 active users and accounts for over 10% of SingPass login activities.
Rethinking how to go big
Going public is a critical period for a digital product to identify and address otherwise hidden issues. However, a sudden explosion in the number of users can be double-edged. It is important to find a balance between driving adoption and scaling to stress test the systems, user experience and the team’s ability to resolve unexpected challenges.
Though the SingPass Mobile team learnt this the hard way, how we were able to respond and subsequently iterate not only the product design but also processes and work flows, resulted in long-term benefits. We highly encourage other teams to take a more progressive approach to scaling and to plan for rapid learning and iteration cycles to take advantage of the influx of user feedback during any release period.
Instead of a singular moment in time, ‘going big’ could mean staging the roll-out over weeks or even months, to ultimately reach every user.
Error 121 has become a hugely valuable lesson for all GovTech teams to discern the difference between being ready to ‘go live’ and being ready to ‘go big’ as well as to maximise the potential benefits from moving more progressively between these different stages of delivery.
SingPass Mobile, as it exists today, is just the tip of the iceberg in our vision to make digital services more accessible and inclusive for all.
To find out more about how we are continuing to identify new user pain points and design opportunities, check out Part 4 of the SingPass Mobile product design journey.
