Besides Functional UI Testing, How Important is Visual Testing?

There are many testing types to implement in testing your software. But software testers sometimes confuse functional UI testing with visual testing.

Let’s take an in-depth look at these two testing types.

Differences between functional UI testing and visual testing

Functional UI testing focuses on testing the UI functionality of the software, whether it works as expected based on product requirements. Software testers will not tolerate any functional bugs on the page, because it will impact many sectors, such as product, data, and even finance.

For example, we want to test functional UI on the payment page that contains checkboxes to select some products in the shopping cart. The checkbox works properly, but the price stated on the payment details does not change according to the products selected. In this case, users will not continue the purchase because the price is incorrect. The company would suffer a great loss due to this bug.

Software testers must be aware of product requirements and execute functional UI testing on every page — even checking page scrolling and integration with other pages.

Meanwhile, the focus of visual testing is on the visual of the software based on Figma or Zeplin. The software tester will validate and manually look over the details of the software design.

The problem with visual testing is we might miss some details when comparing the previous version to a newer one due to human error. And what if there are a lot of pages we need to visually test?

This raises the question: Is visual testing possible to automate?

Implement automated visual testing

Let’s say Black Friday is coming up and your store manager wants to update the register page. The software tester has to make sure that the changes are orderly because it’s a short event that requires many conversions.

In this implementation, I created simple two-register pages using next.js to compare visually according to the case above.

register page production (left) and register page staging (right)

First of all, we need to install:

• python
• pytest
• selenium
• pillow

Now, we set the browser compatibility and default browser in source/libs with the name config.py and configweb.json:

After that, we need a screenshot of the page, so create screenshot.py in source/libs and create a new folder in source/screenshots:

Okay, now we can start to create a test test_register.py in source/web :

Check that our test is run properly python -m pytest -sv — html reports/report.html, the result would look like this:

And in the source/screenshots folder we have two png files.

From the screenshot image, we need to create a grid for each image and create coordinates. Create a new file in source/libs with the name analysis.py :

And adjust source/web/test_register.py :

After the run, a new image grid_registerpage.png would be added in source/screenshots.

grid_registerpage.png

So, what’s next?

We analyze the image region by calculating an average brightness with pixels and colours. Pillow library allows us to access pixels by x coordinate and y coordinate via Image.getpixel(x,y) method which returns 4-tuple of RGBA colour space coordinates.

To calculate an average region “brightness” we will sum up the average RGBA values of every pixel in that region:

So now, we compare the two images’ coordinates and values. If the image has a different value, the result will show it.

When it runs, the result will go from this:

ss_production.png register page (left) and ss_staging.png register page (right)

to this:

the result after compare two-image grid_registerpage.png

As you can see, there are some anomalies:

1. Different titles between the two images.
2. Email boxes have different widths.
3. Found a typo on the submit button.

There are also other alternatives, you can use testing frameworks such as the robot framework which has a library called DocTest library:

GitHub - manykarim/robotframework-doctestlibrary

Or using a python library called applitools eye:

Automated Visual Testing with Python

To access the full code of this implementation, visit here:

GitHub - aziz-alqudsy/poc-visual-testing

Conclusions about visual testing

• Visual testing can impact many sectors that in turn affects the company as a whole. For example, users won’t buy a product because the image is broken or distorted.
• Software tester must be aware of UI changes, especially in responsive mode.
• It’s important to check compatibility in any screen resolution.
• Make sure there are no broken images.
• There could be several bugs caught in visual testing with differing priorities and severity:
  - High priority and high severity example: the products added to the cart of an e-commerce website are not visible on the payment page.
  - High priority and low severity example: the logo on the company’s welcome page is distorted.
  - Low priority and high severity example: some buttons on the website overlap. Although clickable, they may confuse the users.
  - Low priority and low severity example: a spelling mistake on a site page that is not frequently visited.

Hopefully, you can understand what visual testing is about and be more aware of every change that programmers make. Thank you!

Reference:

• https://pillow.readthedocs.io/en/stable/