Visualization testing with BDD — using Applitools
A tool like Applitools used with BDD — really? Well, that was my first reaction when I started working on this task. In my current workplace, we are working on a Java + Cucumber(BDD) Framework. I will not get into the advantages and disadvantages of BDD. There are already quite a few articles on the web which explain this in great detail. This article by sauce labs gives a nice and quick overview. If you do decide to go for the BDD approach, then you must be prepared for a change of mindset. Not only do you need to implement automated tests, but you also need to cater for the BDD syntax. This is the mindset I immediately had to adapt to when taking on such a task.
The following steps were my thought process, of how I thought was best to integrate Applitools to our BDD Framework.
1. Getting to know what Applitools has to offer
One of the reasons, why we chose Applitools over other visualization testing software is due to how well documented the product is. The official website has many videos, blog posts and tutorials to help you get started.
I started by watching the following video:
The video gives a good insight into what the Test Manager is capable of achieving.
I was already in love with the tool, so my next step was trying to figure out how this can be integrated into our framework. The following article goes into more technical detail than the video, showing how easy it is to integrate Applitools into any framework.
2. Preparation
First of all, you need an Applitools API key. Applitools are kind enough to offer this for free for up to 100 validations a month. After creating your account, you can find your API key here:
Copy your key somewhere, as we’ll need it later on.
3. BDD Steps
So we know how to use Applitools Eyes, and we have our account setup, what’s next?
We have to come up with some BDD steps, which will be easy for the end-user to understand and relate with.
Before writing any code, what I like to do is write my tests in BDD format. I do this to prepare myself for how such an integration should work. This saves me a lot of time as I would have thought of all the possible steps before the actual implementation.
A. Initialising AppliTools Driver
The following step is to initialize the connections with the application driver and the Eyes Server
B. Taking screenshots
So the most critical part of Applitools is the actual uploading of the screenshot, right?
So what we have learnt from the video is that Applitools works by uploading screenshots to their service. Each time we upload a screenshot, Applitools will let us know if there are any changes from the last screenshot uploaded. Applitools Eyes will then start learning what is and what is not acceptable, making the tests less flaky and more useful.
This step would entail a screenshot being taken on the current page and uploaded to Applitools. We can make this better though, as explained in the video we can come up with unique names for screenshots for AppliTools to better recognise and compare images. Keeping this in mind I came up with the following step:
C. Closing the AppliTools Driver
The code is pretty self-explanatory. We must let Applitools Eyes know that the test has concluded in order for the service to process the results.
You will also need these two methods:
4. Feature File
The above three steps can be used as shown in the following feature file:
5. Checking out the results
The next step after running our tests is to check out what has been uploaded to Applitools Eyes. On the main page of the dashboard, you should see two newly uploaded screenshots. These are labelled with the “new” mark, as they are the first batch to be uploaded. This is where you have the opportunity to set sections to the screenshots, for future comparisons.
After rerunning the tests, the newly uploaded screenshots are now compared.
You can also set sections, to make your tests more robust:
You can also manually determine the test result from Applitools Eyes:
You will get either “Pass” or “Fail” labels from this point.
Conclusion
There is no denying that Applitools is a fantastic tool, and makes visualization testing more bearable. There is still the constraint of these sort of tests being flaky, and there is not much any vendor tool out there can do. I would still recommend only to implement visualization testing if there is a real business need for it, and that there are enough resources to maintain such tests. Integrating Applitools into a BDD framework was a right way for non-tech team members to use such a tool more efficiently. Making the tests more useful and reusable.
What are your thoughts about mixing the two? Do you like the idea? Leave a comment below and let us know!
P.S. We are looking for guest authors, interested? Contact us.
