How to make your Automated Testing relevant to your users
Why you should have a testing strategy revolving around your customer
To one degree or another, anyone with a background revolving around development has approached the topic of QA and ensuring feature completion to prevent this woeful prospect: A customer has an experience breaking bug that loses money and slows your company growth. The key isn’t to work harder, but smarter. Automated testing is the way and almost everyone sucks at it.
I’m going to just dive into it. Regression testing is likely the first thing you think of when you begin automating your testing. You want to avoid that one little bug you had a while back that dominoed that one bug that took you hours to fix and an automated test when you merge to master would have fixed it. It wouldn’t have, but hey, wishful thinking. Why wouldn’t it have? Because the way you implement regression testing is wrong. Developers can’t tell the difference between a unit test and a regression test and given a unit, regression, or integration test, they get even more confused. In reality, an integration or an end-to-end test might have caught that bug, but who’s got time to build a fully fleshed out e2e test? You do, because future you will thank you. Automated integration and e2e tests are relatively simple to build if you’re building your unit tests as you go like you should be. Testing should follow the clean code prospect of not repeating yourself and reusing code. However, your tests likely aren’t actually fleshed out enough to capture many bugs that your users experience.
Remember the last release planning you took part in? If you’ve experienced a large number of them, you’ve likely experienced the persona method of feature planning. This simple business practice is an excellent method for planning future releases. For the uninitiated, persona planning is a simple thought practice.You start with the basics: What are the different users and how do they use the product? You list out the different users and assign them a name to represent their use cases. Martha the maiden of privacy. Jordan the techie trekkie. Bob the computer-averse virus hoarder. etc.
You take one of these personas, and go through many thought exercises, finding the features, bugs, etc., that matter most to each of them, prioritize these and then develop some user stories and boom: You have a release to build and introduce to your userbase. Read more about it here: https://www.ibm.com/garage/method/practices/think/practice_personas/
What does this have to do with automated testing you ask? Well, if you read this title and got this far, you can probably guess it. You and your developers should probably be using these same personas to identify a testing strategy to give more meaning to your incredibly boring test suite (if you even have that much ha ha).
I want to now walk you through how you can build a test suite that represents and utilizes this method of thinking:
Situation:
Your business builds an app targeting people who love to journal.
The Users:
Karen, the mother of 7 who loves writing fanfic for supernatural and true crime and dreams of having a hit book someday.
Bruce, the dog-mom who won’t stop talking about his dogs and misses his past beloved pets.
The Test Planning: (Yes, plan your tests.)
As a team, you would discuss the two above users and how they may use the app. What would catch their eye? What would they tap, interact with, upload, download, view, etc. You build situations and think of real-life interactions. You may use real-life data collected by analytics or customer support, use past knowledge of user-reported bugs, etc., to flesh out several use cases.
Karen, at the end of the day, she relaxes by watching an episode of supernatural, and when she opens your app rather than talking about her day, she talks about the episode, and rereads past writings on other episodes. She makes up a writing prompt and rather than using your app to save it and organize it, she shares it to her facebook fanfic group. Your company already addressed how to capture this use case and has a feature set planned out and development is underway. You then create e2e tests that go through the above interactions, putting on the Karen mask in order to tap on the wrong places and ultimately discovering your new feature and rather than simply just ensuring it works, you truly put on the Karen and get into her head to build the test.
Bruce, at the end of the day, takes a dozen photos of his dog that basically look the same but don’t to him, and uploads them to your app for the day. He then wishes he could see all the entries about this dog. Company has already captured this feature, and developed ML feature that groups journal entries that are visually similar and now you have to test that the feature works and more than that, actually works for the customer’s desires. You create integration tests that go through the above interactions with your app, uploading dozens of similar looking images and rather than looking at them individually, you jump to past days based on suggested images.
The Outcome:
You catch more bugs. Turns out Karen’s extra taps cause the journal submission to be scrollable when it shouldn’t be because formatting elements are getting activated by the awkward tapping this persona does. Bruce apparently has been experiencing some garbage data from the ghost of pages past (from all the old entries he’s been visiting) and this is breaking the suggested content area. The exercise of fleshing out your e2e and integration testing by adopting a very specific persona has enabled you to create tests nearly every time that accurately (probably) represent a user, in order to actually catch bugs that users experience.
Persona oriented automated testing is not meant to replace your testing strategies, but to add depth, meaning, and to flesh them out. Putting meat on their bones, so to say. Expanding and building on the wireframes you typically develop is an important aspect of QA, it enables you to cast a larger bug net. Not only this, but building the test personas required you to work closer to others in your company. You had to discuss users with marketing and customer support. You had to learn about actual users and developed fake users from the information you learned. The result is a more integrated QA team, more communication, and better tests in general. Your job is more secure, your value better represented, and development is more in touch with the customer.
