How 3 Process Patterns will change the way you do Test Automation

Uladzislau Ramanenka
Apr 27 · 12 min read

A few years ago, here at Bumble — the parent company that operates Badoo and Bumble apps — we started to shift towards using autotests more in our development, testing and release processes. We have, however, encountered a few challenges along the way.

Before we start, you might be curious to know why we have been investing in Test Automation in the first place. The answer is simple: our mobile applications have weekly releases. For the QA department, this means we need to test the feature itself, integrate it into the application and do the regression. Effectively, complex regression testing must be performed at least once per week, which is frankly exhausting.

Test Automation was our response to this situation and was intended to make releases faster and more reliable (read the full story in more detail here).

As you know, when trying to resolve an issue in software development we often use a pattern — a general, reusable solution to a commonly occurring problem within any given context. Test Automation is no exception, there is even a handy Test Automation patterns wiki reference. In this article, our main focus will be on Process patterns for Test Automation. As their name suggests, they show how the Test Automation process should be set up or how it can be improved.

Without any further introductions, here are the issues that we’ve struggled with the most and some recommendations on how to overcome them.

Ask for help

Ask for help

Initially, our QA team wasn’t very confident with automation tests. As often happens in engineering, our QA team is diverse: we all have different backgrounds, experiences, and domains of expertise. It’s important to highlight that back then not everybody in the QA team had been exposed to Test Automation. When we decided to invest more in autotests, it was a steep learning curve for some of us. To help our QAs overcome their new challenge, we encouraged them to ask the Test Automation team for help. The reasoning behind this was that if you get stuck on a particular problem, you shouldn’t ponder too long on it — others may have faced the same issues and so perhaps there’s already a solution. Still, in order to avoid overwhelming the Test Automation engineers with loads of requests, we decided to formalise the approach. The ASK FOR HELP Process pattern seemed to be a wonderful route to go — Ask for help instead of wasting time trying to do everything yourself. After all, no single team is going to have all the skills needed at any one time but instead, having a pool of experts to turn to, is the way to go.

As it turned out, merely having a pattern to follow can cause problems. The Test Automation team started getting even more questions than usual (for example, on issues around environment setup). Over time, we often noticed the same questions were being asked by different team members. It was taking significant time on the part of our Test Automation team to respond to all those people because even a simple well-understood issue requires context switching.

However, for most cases, we do have documentation and solutions for common issues on our local Wiki. It may be not the best way to document all possible errors and explanations but we thought it would be useful to have them stored somewhere.

So, we came up with the idea of local StackOverflow. Our current solution is based on the Open Source solution — Scoold. It is designed to act as the first line of inquiry. It works like the common StackOverflow but is limited to the people inside the company. Whenever somebody has a problem, it will be enough to use the local StackOverflow: either to find the answer or to post a new question and get a reply from the expert team.

This is what our local StackOverflow looks like

Potential pitfalls:

  • No responsible person to kickstart the implementation. Even the simplest solution requires somebody to drive it through. Luckily, we had such a kind of person. It’s equally important to ensure that management both understands and supports the initiative.

Benefits:

  • Process gamification, so more people are involved in resolving issues. Note, that we didn’t invest heavily in different badges or other perks. In fact, the number of upvotes or stars a person gets doesn’t mean anything. We believe you shouldn’t cross the line here. Gamification is only an additional stimulus for participants, not the main actor.

Set standards

Set standards

Whenever possible we seek to share code across the apps/platforms to facilitate maintenance and support. But as with any other system developed over the years by many contributors, different parts can be written in different styles. Our main language for end-to-end autotests is Ruby (to explore the history of Mobile Test Automation in our company click here). In Ruby, intentionally, there are several ways to do the same thing. This is because Ruby’s creator Yukihiro Matsumoto wanted developers to use whichever way was natural to them to get the job done. Furthermore, as you remember, we all had different previous experiences and so might have coded in different ways. Consequently, these all meant it may be challenging to work on someone else’s code.

SET STANDARDS process pattern — Set and follow standards for the automation artefacts — was chosen to deal with the problem. Without that, it might be that many people work on the same project, but all use different styles and methods (imagine they “speak different languages”). For this reason, they are unable to share their work efficiently. We started by gathering feedback from the individuals who would be using the automation framework and tools. In this way, we could ensure that the conventions we adopted would serve all the users. Moreover, anyone involved in the decision-making would be sure to be more supportive of it in the future. You wouldn’t like having some arbitrarily imposed standards if people don’t understand the reasoning behind them. Thus, we’ve worked up the common guidelines that are shared and adhered to by all team members and helped us to find a common language between us all.

What we have done:

  • We have set up a local wiki with up-to-date guides and documents (i.e. how to set up the environment, run the tests, and support the tests), which also help shorten code review cycles. We use Confluence for all documentation and have split it into a few spaces. All pages are periodically reviewed and looked after by their respective teams.

Potential pitfalls:

  • Documentation once created is not maintained. This is a known issue for any type of documentation, especially when it’s not in constant use. Our solution is to appoint maintainers for the docs and to be sure the notifications that Confluence sends to the maintainer’s inbox are not going to be ignored whenever somebody leaves a comment.

Benefits

  • Less time and effort required for tests maintenance and bug fixes. Having references to guide you while writing the code makes it much clearer what to write and how to write.

Share information

Share information

As a team, we are trying both to learn concepts and enhance expertise so we encourage everybody to share their knowledge with others. This could be a root cause analysis of the tricky bug; a case study; a sneak-peek into instrumentation internals; or anything considered to be potentially beneficial to the others.

This pattern can be described as SHARE INFORMATION pattern — Ask for and give information to managers, developers and other testers. For us, this is the way to benefit from a wider pool of knowledge that will improve Test Automation.

Thus, we’ve launched weekly QA lightning talks. Any person can suggest a topic and present it for 10–15 mins, with Q&As to follow. Because meetings are brief and time-limited, there’s enough time to learn something new without disrupting too much of the day. If anyone misses a presentation, there’s still a chance to catch up later, as all recordings are stored in the internal library. Lightning talks are not limited to Testing and Test Automation only but can be on a broader topic. For instance, once our colleague talked about their experience of being a Homebrew maintainer. One of the topics I personally have presented was about me being a volunteer ‘armchair mapper’ for Humanitarian OpenStreetMap. What I do there is create maps for the poorly mapped areas. These later get used by various organisations such as British Red Cross or Medecins Sans Frontieres in their fieldwork. A shortened version of this talk was later featured in the SoapBox session at the Eurostar 2020 conference.

Potential pitfalls:

  • Not everybody will be ready to prepare and present their talks. Indeed, not only does it take time to prepare a presentation, but the presentation to others also requires extra effort and courage. We tackle this problem by restricting the length of talks and offering plenty of help and support to anybody interested in being a speaker. We’ve already got a bunch of people who have made their first steps in speaking.

Benefits:

  • Improve work visibility among the team members. The point is that, occasionally, and this may sound familiar to the companies with many teams, we might inadvertently end up ‘working in silos’. Therefore, lightning talks can be a valuable addition to the actions you take towards developing a more collaborative way of working.

Bonus: Pair up

Pair up

One further point I’d like to mention is something that resulted from the previous steps that we’ve taken. Namely, one outcome was that QAs have become ready to take (and gradually taken) ownership of writing, maintaining and analysing the failed automation tests.

So, the intention now was to go a step further — with QAs taking over the Test Automation Framework and some tools related tasks, while the Test Automation team moves towards more Research, Innovation, and building internal testing tools. Such a shift would be impossible if QAs failed to learn the framework, test inner specifics or did in-depth work with the codebase — beyond their day-to-day tasks. Regular activities were being done to a decent standard, but it seemed that going beyond them would not be easily achievable without help.

To achieve our objective we’ve used the PAIR UP Process pattern — less experienced team members pair up with more experienced team members.

The result is that we’ve launched Internal Mentorships for QA Engineers within the Test Automation team. What exactly is it? QA engineers join the Test Automation team for one sprint, usually a 2-weeks period, and work on the tasks from the Test Automation team backlog, not on their own, but with help and mentoring from a Senior Test Automation Engineer. The main idea behind internal mentoring — is learning. To highlight this, we’ve devised the following manifesto which listed all goals and expectations:

  • Focus on learning over strict time-bound delivery

While there is value in the items on the right, we value the items on the left more.

More information on internal Mentorships, why and how exactly we do them, can be found in my recent talk here.

Summary

In conclusion, to resolve a commonly occurring problem in software development it’s usually a good idea to investigate patterns in the software design. As for us, by adhering to the Process patterns for Test Automation we were able to resolve the following challenges:

  • Reluctance to seek aid from peers (solved using ‘Ask for help’ pattern). As Seretta Gamba and Dorothy Graham point out in their book “A Journey through Test Automation Patterns: One team’s adventures with the Test Automation“ — “Don’t be afraid to ask for help: most people actually enjoy helping”;

We believe that following these patterns is crucial in making your Test Automation process efficient.

Having listed the pros and potential pitfalls of the selected approaches, we also wanted to show how you can apply the patterns.

In this article, we have only shed light on some of the Process patterns for Test automation. But we hope it will serve as an inspiration and you’ll benefit from following our recommendations.

Please don’t hesitate to share in the comment section the issues you faced in your Test Automation journey and how you resolved them. We would love to hear from you.

Bumble Tech

This is the Bumble tech team blog focused on technology and…

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store