Life Lessons in Usability Testing
How testing Column.us taught me to seek fresh perspectives and celebrate what makes each of us awesome
Thus far in my life, I have found that whenever I am allowed to assess something, from music to another person’s cooking, my mind immediately leaps to the myriad ways I can break their effort down and build it back up in my image, suiting my particular taste. Am I a narcissist? Anyway, the point is that I love to analyze! I imagine that most UX Designers also relish the opportunity to dive deeply into a project and find gems.
You can understand, then, how thrilled I was to be given the assignment to assess the usability of Column.us, a website that pledges to deliver a streamlined experience in the creation of public notices for local newspapers across the country. Column found itself seeking feedback in the arena of usability and decided to utilize an external team.
“Amazing”, I thought. “I can’t wait to get started! Also…what’s a public notice?”
Research
To Google I went, in the hopes of understanding why public notices mattered. The answers I received were legitimately surprising. This piece from Nieman Reports, a publication “covering thought leadership in journalism” describes public notices as a cornerstone of democracy, scaled down to the microcosms of local and state governments. They are critical to holding leaders in our communities accountable, which isn’t possible without their actions first being made transparent. Public notices provide a way for individual members in the community to not only stay informed, but alert other members to important legal, business, or personal matters happening in real-time. Finally, they provide an invaluable revenue stream for local newspapers, which has become increasingly pressing with each passing year as readership dissipates.
Public notices may seem like simple ads in newspapers, but they are so much more.
Heuristics Evaluation & WCAG Audit
Armed with newfound knowledge, I set out to delve into the usability of Column’s product. First on the list was performing a heuristics evaluation on the site, based upon Nielsen Norman Group’s Heuristic Principles. There were specific principles that were the most pressing to address, including a paucity of help & documentation for individual users, little to no instructions, guidance, or explanation on the site as to what a public notice is, how to write one, or what to include in it, and a lack of confirmation upon submission of one’s notice. Another evaluation I ran was a WCAG audit on the website to assess its level of accessibility, and there were a few items I noted. First, nearly every page had the same page title- “Column- Public Notice”, which would render individuals utilizing screen readers confused as to which page they were currently on. Next, I realized the Help Center page was missing alternative text for images, which would make it difficult for individuals who rely on those descriptors to navigate. Finally, the color contrast on several pages on the site was below the recommended ratio of 4.5:1.
Usability Testing
My heuristics evaluation gave me ideas of what to hone in on when faced with actual users. Because of the one-week window of opportunity I had, I was encouraged to complete my first round of usability testing with people in my social network. I rounded up 5 individuals for 5 separate, moderated tests via Zoom. Participants shared their screen with me as they walked and talked their way through the Column website upon my prompting. Some of the questions and tasks I included during this informal testing were:
Can you tell me, in your own words, what this website is about and who it is for?
Can you explain to me if and/or how this website explains to you what a public notice is?
Walk me through the process of: registering for an account, posting a notice, confirming a notice, seeking help on the site, and exploring your user settings.
My second round of testing was completed with the help of usertesting.com. I was able to obtain 4 individual testers, and through a video recording of their screens via the testing website, I completed unmoderated testing thanks to a series of tasks and reflection questions I wrote and then asked testers to complete and answer. They included the following:
Without clicking on anything and staying on the first landing page, can you tell me what this website is about and who it is for? What stands out the most to you, and why?
After completing the step above, feel free to peruse the site via the labels at the top. Upon doing so, can you tell me if the site does or does not explain what a public notice is?
Upon your account registration, please walk and talk through the process of publishing a notice in and receiving confirmation from Alaska’s “Eureka Herald” publication. The notice should consist of your intention to seek a mixed beverage liquor license for your restaurant, The Spotted Poppy. Afterward, please walk and talk through the process of navigating back to your dashboard.
After posting your notice, please walk through how you would look for instructions on the help page regarding how to use the public notice templates when creating posts.
Please navigate to the settings page. Does anything surprise you on this page? Please talk through what you notice here. After that, please walk and talk through the process of changing your email address and password on the settings page.
After completing these tasks, I asked the testers to discuss how they felt upon navigating each stage of the site, their confidence that the site would publish their notice as desired on a 5-point scale, and the likelihood they would recommend the site to others on a 5-point scale.
Findings, Insights, & Recommendations
For the most part, much of the feedback on the Column site didn’t surprise me- generally, the users’ points largely matched my own from initial evaluations. A few quotes, in particular, echoed sentiments felt nearly across the board:
“I felt like I was back in school, trying to figure things out” -regarding the site
“It’s important to assume I don’t know what a public notice is or why it’s important”
“Help Center pages do not seem catered to individuals”
What delighted me, however, were the comment finding several noteworthy attributes of the site, and their affinity amongst the majority of testers. They included: the straightforward registration process, the interface of the notice dashboard, the actual process of creating the notice and putting in the information, and the logo design and overall sleekness of the site.
I realized that distinguishing and elevating what works while usability testing can lead to important breakthroughs.
I created an affinity map to easily assess the patterns in the plethora of responses I obtained.
After putting together oft-repeated findings from my users, I synthesized them into a persona that anyone (hopefully the Column team!) might deem useful in further assessing the usability of the site.
Meet Samuel
Creating the persona of Samuel was an excellent way of contextualizing the findings of my research. But to truly understand and thus communicate the users’ experiences and pain points across their interactions with Column’s site, I created a journey map consisting of his use case: Samuel is currently pursuing a Type 47 liquor license in California for his new restaurant, and that specific license requires the publication of a notice in a local newspaper of general circulation in the nearest town to the business’s premises, under Bus. & Prof. Code § 23986.
While there are a million and one ways to break down any product or service, I wanted to use Samuel’s journey to illustrate the most critically important findings, conveyed near-unanimously by all my testers. With each finding, I challenged myself to ask “Why?” several times in succession, until arriving at a conclusion I believe served as valuable insight. Only then did I make recommendations based upon those insights. The following is a list of actions I have suggested to the team at Column after conducting my usability tests:
Finding: Users could not glean the purpose of the site, even after looking at the About page, Solutions page, or any of the pages in the Help Center.
Insight: Why? Because they could not or would not read through every page closely. Why? Because the text on every page was lengthy, seemed convoluted, and highbrow. Why? Because sitewide, there is more of an emphasis on extolling the myriad benefits of Column as opposed to providing simple explanations regarding who and what the site is for.
Recommendation: Clearly define and provide concise explanations of what you do as a site, and for whom, on every page, and keep it consistent across all stakeholders. Ensure that the interpretation of your services is not left to chance from users.
Finding: Users did not find information in the Help Center or any other page legitimately helpful in attempting to understand how to utilize the site
Insight: Why? Because they found information that catered specifically to Publisher, Advertiser, and Donation Campaign roles, and they did not identify with any of those stakeholders. Why? Because individual users are their own distinct entity, have little understanding of the site, and want their needs attended to.
Recommendation: Attend to the needs of your individual users, and make individual users a key stakeholder of your site by communicating how they can utilize it (as opposed to publishers, legal entities, government agencies, etc). Restructure the Help Center to include information that is specific to their accounts.
Finding: Users expressed uncertainty about what to write when creating their trial public notices until told specific information from the test giver.
Insight: Why? Because most users, before participating in usability testing, were unaware of the existence of this form of communication and therefore unfamiliar with the guidelines for creating a public notice and the information they are required to contain.
Recommendation: Provide public notice templates and examples for every state and publication. Make it clear that there are categories of templates and explain the purpose of each one.
Final Thoughts
I’ve learned so much over the past week thanks to this project. Not only have I become a better UX Designer because of it, but testing the usability of a product has reminded me of my own values and hard-won life lessons.
Usability Testing teaches us that what truly matters is what we do and not what we say. As a designer, it was critical to watch how users were utilizing this product.
Oscar Wilde said, rather cynically (it was kind of his thing), “Most people are other people. Their thoughts are someone else’s opinions, their lives a mimicry, their passions a quotation.”
While I have a slightly more optimistic appraisal of humans, I do think he speaks to a universal phenomenon. People think of themselves as other people, greater than or less than who they truly are. This personal illusion carries over into testing; as UX practitioners, we lose the opportunity to meet users where they actually need us to be when we accept everything they tell us at face value instead of observing them exactly as they already are. It’s a reminder that we are the most valuable human beings, with myriad compelling insights to share, when we abandon all pretense and commit to being our truest selves.
Usability Testing has also reminded me that it’s always commendable to ask for another person’s perspective when you need it. Never fool yourself into thinking you have all the answers, know when to collaborate on the problems you’re trying to solve, face the music when you are confronted with things to work on, but also don’t forget the wonderful things you have going for you so you can continue building upon them! UX Design truly is life- who would've thought?
Thank you so much to the team at Column.us for this opportunity, and thank you all for reading!
