Why visual testing is more than catching bugs
Catching visual bugs is the #1 use case most people think of when they think “visual testing.”
Although that’s true, there’s so much more to it.
In this post, we’ll see how visual testing is about getting fast and accurate visual coverage, visibility, and confidence in three scenarios:
- When visual changes are bugs
- When visual changes are intentional
- When no visual changes are detected
In all of these scenarios, visual testing helps teams build quality products and deploy with complete confidence, without having to slow down for manual QA.
When visual changes are bugs
Visual testing is certainly the best way to catch visual bugs before they make it to production. Catching bugs is the most common reason that developers start doing visual testing in the first place.
Before adopting a full visual testing process, developers often try introducing specific tests for things like colors, styles, or other visual elements — only to find that bugs keep slipping through, and tests become much harder to manage.
Unit and functional testing, while very useful and important for healthy software, are simply not the right level of abstraction to test visual changes. Trying to write visual regression tests in functional tests quickly becomes a slippery slope of fragile assertions piled on top of other fragile assertions. With a codebase full of brittle and unmaintainable tests, your test suite ends up slowing down development instead of building quality software quickly.
Visual testing tackles this issue at its core by examining the visuals themselves rather than the code underneath, and surfacing bugs before they ever reach users.
By generating and comparing browser snapshots to see what pixels have changed, you’re actually testing what your users actually see and interact with — without manual QA.
This kind of automation allows us to get full visual coverage in a world where browsers are complex, CSS is hard, and there are too many combinations of devices and screen sizes to manually test with 100% accuracy.
When visual changes are intentional
Visual testing helps teams quickly catch regressions and bugs, but what about when visual changes are deliberate?
When integrated into your pull request and code review workflow, visual testing gives you a full review process for visual changes. No need to put “before” and “after” screenshots in your PR descriptions — which not only quickly become out of date, they also easily miss the full scope of changes throughout your app.
Visual testing gives you an automated workflow on every commit to see the exact visual changes being made across your application in every state.
That visibility gives your whole team insight into what has changed and what about to be shipped. Whether it’s a developer making sure a new feature looks right, a designer checking to see their designs have been correctly implemented, a product manager staying in the loop, or a product marketer grabbing updated UI screenshots, visual testing provides that insight and coverage.
When no visual changes are detected
Although not as overt as our previous two, this last scenario is one of our favorite reasons for doing visual reviews on every pull request.
Often when making software changes, it’s the fear of unknown bugs that causes the most stress.
Could this break something?
How will I know if it has?
What other changes might ripple out from this?
This fear is often why we spend time manually testing changes in the first place. Visual testing, on the other hand, gives you complete certainty that you know the full scope and impact of your changes, including things just remaining stable.
If you’re told that 100 pages are unchanged, that’s 100 places you don’t have to check. In our experience, this is a huge confidence and speed boost for modern software teams.
There are several situations when it’s invaluable to know that your UI remained stable, like deleting/refactoring CSS or upgrading dependencies. Without visual testing, those are tasks that developers lose sleep over or just don’t do at all, or spend countless hours manually checking every page in every browser and viewport size.
Visual testing automates that work with a precision not realistically possible before, empowering you to seamlessly merge and deploy with full confidence that the visual side of your app has remained stable and exactly as you intended it to be.
Visual testing is more than just catching visual regressions. It’s about having complete confidence in what your users see and interact with, at a higher degree and level of scale not otherwise possible with other testing methods.
At Percy, we’re building our visual testing and review platform to give teams a new way to quickly iterate and build software with confidence. We hope our easy-to-integrate SDKs, purpose-built rendering infrastructure, and free plan make it easier for teams of all shapes and sizes to reap the benefits of visual testing.
Happy testing! 🕵️♀️
