What we changed about our platform when we listened to our users
Accreditron is a “multi-stakeholder” software product. When designing Accreditron our team has to think about every little interaction or edge-case. How does someone sign-in? What should be the first thing they see? What happens if we put this button in the bottom-right versus top-left? What if someone tries to click the page 500 times a second?
Background
We’re a company of designers, copy-writers, user-researchers, talkers, and software engineers. We prioritise design, full-stop. Moreover, we think that pleasant and purposeful user experience is a key offering to the stakeholders and users of Accreditron. A stakeholder being someone who has interest in the product versus a user being a person who will actively engage in using the product. Our stakeholders are spread across different levels of New Zealand Government and the Social Sector. Every person comes from a different background, they think differently from each other, and they all use computers in different ways.
We think Accreditron should be zero cost to the social sector. This means, like many enterprise software companies, the people funding Accreditron are not representative of all the users that will engage with the product. Instead, funding stakeholders might be the management, government leadership, or funding organisations. Because of this difference in incentives, a disconnect can form between what the funding stakeholder wants and what the users need.
Usability research is an essential contributor to our success and why we take design and evidence-based decision making seriously. It also allows us to keep the voice of all users at the table and supports us to build our product directly for the people using it. It’s all about collaboration and communication.
Usability Research on Accreditron
Usability research is the process of researching and interviewing people who interact with Accreditron. From gathering broad data on their problems or journey — to detailed research like watching a person perform a task to understand their thought process.
For every component on Accreditron, we first research why we should build it and test our assumptions. Then after we design or build something worth testing, we research and measure how our solution performed against our original assumptions. With those results, we can make changes to improve the experience and repeat the process. This is sometimes referred to as the build-measure-learn cycle.
Usability research is one option for gathering data and measuring performance. By holding many usability sessions and interviews, we can pull all of the information together to develop insights across a sample group. Then use these gathered insights to drive the design of Accreditron and provide validation where possible.
In December of 2017, we finished up a round of usability research with social service providers in New Zealand. We heard heaps of positive feedback about Accreditron like:
“It’s really nice and clean, the set-up of it is easy to use.”
“I really like it, it’ll make the process so much easier.”
“If we just used Accreditron it would be wonderful!”
How We Ran Our Usability Trial
Process: Our usability trials consisted of a short pre-trial interview to gather general information with a business admin person from a social service provider. Then we asked the interviewee to perform a task on our pre-release alpha version of Accreditron followed up with a short post-trial interview and questionnaire. Our goal was to gather both qualitative and quantitative data on user experiences for a social service provider using Accreditron.
Sample Size: We conducted trials with 12 different social service providers. Generally, the recommended sample size for usability trials is about 5 or more for each segment. The social sector has an incredible range of organisations — from small after-school care organisations with one funding source to large nationwide counselling services who work closely with multiple ministries.
We specifically focused on testing a range of organisation sizes — large organisations have entirely different internal structures and processes compared to small organisations. We had to make sure our product worked from small, to huge.
Task: All users were given the same task as part of the usability trial. By offering the same prompt to all participants, we ensured that their responses were as unbiased as possible as well as more directly comparable. The task providers were asked to perform was fairly straightforward:
“Using the Accreditron platform submit your pre-assessment.”
And with that, they were let loose.
Testing Our Assumptions
Below are four assumptions that were tested along with some of the learnings we obtained from the December 2017 usability trials of Accreditron.
1. Login and registration
Assumption:
Users will be able to login and register their organisation on Accreditron.
We implemented a passwordless login process to improve security and prevent that pesky problem of forgetting your password. We know from experience that some of our users are likely to use the same password across multiple devices which is a security risk to Accreditron. We designed around that: when you sign into Accreditron you enter your email then check your email for a link that directs you into Accreditron as an authorised user. Even though passwordless login systems are commonly being implemented across the web, we still wanted to check that social service providers intuitively understood the process.
Finding:
None of our users got stuck and they found the process easy to understand! While the passwordless login was a new experience for a number of the trial providers, the interface and simple directions on the initial screen for Accreditron was easy for them to follow.
Result:
This is a real win for Accreditron; it lets us know that users’ first interaction with the platform is not only intuitive but also that it is approachable enough for users to be guided through new processes without unnecessary frustration.
2. Setting expectations at the start
Assumption:
Social Service Providers will use Accreditron to submit the Accreditation Pre-Assessment for review to an MSD Assessor. At the beginning of the process, we want them to have their expectations set so they have no surprises.
Before providers begin their pre-assessment there are a few things we want to convey: their pre-assessment auto-saves, they can fill it out in any order, and they should reach out to their assessor if they need help. Because information will auto-populate into form fields on Accreditron they will still need to verify and check that all information they are submitting has been reviewed.
We assumed that a provider would tick to verify their information as they finish each section of the process.
Findings:
For the majority of users, the way expectations were set at the start was effective with a number of people even getting pretty excited about our auto-saving feature. However, we found that a quarter of users either didn’t make use of the verification system or only did so sporadically. One of the providers who consistently verified offered some insight: she suggested that the box that you click on to verify might need to be more obvious and that might improve her experience.
Result:
Our approach to the potential issue that verification presents has been two-fold: we’ve re-designed our on-boarding page to make it more engaging so that expectations are even clearer at the beginning of the pre-assessment. We’re also redesigning the verification box so that it’s more obvious to users and encourages them to interact with it.
3. Questions and content
Assumption:
Users will find the process and content such as questions and functionality engaging and easy to understand.
Accreditron is made up of questions. By using simple, easy to understand language we can help to make sure things go smoothly for everyone involved. The same goes for functionality, things like which buttons to click and how they work. We wanted to know what made sense to users and what could be clearer so that we could figure out how to offer the best possible experience.
Finding:
With so many different providers from a range of different backgrounds making use of Accreditron we’re highly aware of the need for it to be as accessible as possible. While most of our questions and functionality made sense to users the use of specialised terminology was highlighted as a potential pain point by a number of users. For example on a question referring to provider operated programmes as services one provider noted that it was possible: “they wouldn’t think of their programmes as services.”
Result:
What this shows is the importance of having questions that are as straightforward as possible. While the overall response to our questions was positive we want to perfect the experience and make it as accessible as possible. To address this we have taken onboard provider feedback and have been working with Ministry of Social Development and Statistics New Zealand to make sure our question set is as practical and easy to understand as possible. When we’ve finished this process we’ll also be going over our questions again making sure they’re all in plain English.
4. Navigation
Assumption:
Users will be able to navigate their way through Accreditron and will find the process intuitive both conceptually and practically.
Computers are becoming more and more commonplace in our daily lives, the thing is, not everyone uses them in the same way. Understandably navigating through Accreditron is crucial to its use, however, making sure that it is easy to do so can be difficult; what makes sense to one person might not make sense to someone else. Finding out how people interact with our current navigation system lets us know if it’s intuitive but also if it’s possible to optimise it for use by as many people as possible.
Finding:
Overall, navigation proved to be a pain-free experience for most users. There was one finding in this area however that proved to be of particular interest to the team here at Accreditron. Because Accreditron collects a lot of information, we sometimes hide forms within larger forms that only display when you click the card to expand it. When one of our trial participants was filling out fields within a card, they would use the scroll-bar on the side of the screen to move down the page. Because this involved clicking outside of the form card, the card would then minimise thinking the user was intentionally closing it, making it feel difficult to continue with the task.
Result:
For us, this was a perfect example of why usability testing is so valuable to the design process. We hadn’t encountered this issue before as we all used our mouse scroll wheels or two fingers on our trackpad to move down the page. Thanks to usability testing and the generosity of providers in giving us their time we were able to pick up on this potential issue and make sure it wouldn’t happen again.
Overview
The response to Accreditron was overwhelmingly positive, to put this in perspective one of the questions in the final questionnaire asked participants to respond to the prompt: “I would opt to use Accreditron rather than the current process.” 100% of participants agreed with this statement with eight indicating that they “Extremely Agreed” the highest possible option. Beyond this we also received valuable feedback on how we could improve our platform in various ways, through function, language, and design.
Next Steps for Accreditron
Accreditron is on a mission to reduce duplication of effort across government agencies and the social sector. We look at every conversation as an opportunity to learn, and well… be better at what we do. We appreciate the support of the agencies and social service providers who have been coming along on the journey of developing Accreditron by participating in co-design sessions and usability trials.
The learnings that are listed above and more will be incorporated into the next release version of Accreditron. To find out more about the status Accreditron’s alpha release or how we are running the usability sessions with government agencies reach out at hello@accreditron.com or visit our website at www.accreditron.com
