Why is manual regression testing still happening in 2023?

Is manual regression testing still relevant considering the multitude of automated regression testing tools available?

Are you tired of reading the same old advice on automation vs. manual regression testing? I know I am. I bet you’ve seen people ask similar questions on Stack Overflow. But, before you click away, hear me out.

I have a compelling reason to add my two cents to the discussion. Most manual regression testing tutorials will tell you that the complexity of manual testing grows exponentially as your project scales. It’s true that automation seems like the clear winner on paper. Automated tests are faster, reduce human error, and are cost-efficient in the long run. But, in real-life situations, things are not that simple — especially for small projects with a tight budget.

And this is even more true for UI testing. So, before you go all-in on automation, consider these scenarios where manual regression testing may be necessary. It can save you time and money down the line, and it could be the difference between a successful project and a failed one.

Don’t take my word for it — keep reading to find out why.

Small teams that lack dedicated QA automation engineers

It’s incredible to think that Amazon, Apple, and Google started as small garage projects. Even today, many tech companies often start lean, with 2–4 developers per tech stack.

This often means that less skilled team members are responsible for regression testing, including non-technical staff like project managers or business people performing UI-level regression tests. Unfortunately, manual testing cannot be completely eliminated from the process.

The issue with manual regression testing is that it takes a lot of time, especially when it comes to UI testing on an emerging project. Not having a dedicated tester to oversee these iterations means risking missing important issues and making regression tests superficial. This could lead to delays and put the project at risk.

Exploratory and ad-hoc testing

Manual testing is a crucial part of software development. It’s not just small teams that need it; even full-manned teams can benefit from adding occasional manual tests to their process.

One approach that forward-thinking teams are adopting is adding manual exploratory UI testing to their regression testing cycles. This approach involves non-QA staff, such as programmers, designers, project managers, and even marketers. This might sound unconventional, but it’s a proven way to discover fringe cases and usability problems.

Adding new cases to regression suite during the software development process, known as ad-hoc testing, can be a game-changer for producing high-quality software, if done occasionally (at least). However, ad-hoc testing is not without its challenges. It doesn't follow a formal process, making the tests difficult to replicate. Besides, adding new cases to the regression suite mid-process means spending more time.

Automating regression testing is great, but it’s not enough. You need exploratory and ad-hoc testing to detect critical bugs and expand your testing suite with valuable tests.

Visual regression testing

If you’re struggling with visual regression testing, you’re not alone. Many teams face this challenge when relying on traditional methods and code-based automation tools.

The truth is that these tools can’t always catch errors that are obvious to the human eye. Imagine this scenario: one of your developers accidentally changes the color of a button. Even though this change is visible to anyone who looks at the UI, automated tests that target the button by its HTML tags, IDs, class, or position in the DOM might miss this error. By the way, check Eric Ries’s book “The Lean Startup” to find this issue described there.

Unfortunately, similar cases are all too common. An update to your codebase can unintentionally rewrite a LESS or Sass variable, leading to a change in color, margins, or element positioning. An accidental alteration of the z-index can make a pop-up disappear…

There are many automation tools available to address these issues. However, most of them only offer basic screenshot comparisons, which isn’t always enough. In fact, many companies find themselves resorting to manual regression testing after experimenting with these tools.

The bottom line is that one of the most unstable components of our projects is the UI. To ensure that our teams produce the best possible results, we need to have a solid plan in place for visual regression testing. As of now, it seems that the best tool for this is still the human eye.

So what’s the right mix between manual testing and automation tests in 2023?

As we approach the end of 2023, finding the right balance between manual testing and automated tests is becoming increasingly important. Although manual testing may seem like a practical option in most cases, it may not always be the most efficient one. On the other hand, the traditional code-based approach to test automation, while cost-effective, can sometimes have its limitations.

An AI-powered record-playback solution can be the solution to your testing needs. It can help expand regression suites by discovering tests from exploratory and ad-hoc testing. Additionally, with a solid set of self-test-editing features, running and maintaining visual regression testing can be effortless. However, it’s worth noting that record-based solutions for test automation can still be cumbersome, especially when it comes to visual testing.

So, is manual regression testing still relevant in 2023? Absolutely! However, you can always automate it to make the process more efficient without compromising on quality. By choosing the right tool, you can achieve the perfect balance between manual and automated testing, ensuring that your testing process is both efficient and effective.