UI Automation: Our first approach
Some months ago we started the creation of an automation framework to create and maintain automated test cases for our web system. All of us (as members of the QA Team) are convinced that we learned a lot about this experience, and we think that the best way to share all the acquired knowledge is… through this article! We hope you find it useful :)
What is UI Automation?
First of all, we need to define what Automation Test is. It is an automated way to execute tests on a technological solution that could be part of a system. UI Test Automation is just a part of test automation.
Automation Testing has a lot of pros and cons because it is a tool that can be useful in some specific situations:
Automation is good for:
- Making manual test cases repeatable.
- Reduce error prone manual execution.
- Speeding up tests execution.
- Execute several instances of same test with different data in less time.
- Execute regression testing.
- Execute using different environment and browsers
Automation is not good for:
- Exploratory testing.
- Very complex business rules.
- Test that will be executed only a few times or are so basic that the effort to automate is not worth it.
We need to remark the last point, because even if an automated test case could save 4 hours of “manual testing” for a specific test case suite that your team runs once a year, you could spend 30 hours automating that test suite. In that case, it isn’t worth it.
The next graph will help you decide when you should automate your test suite or not.
To write our UI automated test we used a tool called Cucumber, that allows to write the test cases using natural language.
Cucumber reads the specification from plain language files, called “.features”, reviews them for scenarios to test and run the scenarios against the system.
Each scenario is a list of steps that need to follow some basic syntax rules, so Cucumber can understand the feature file.
Gherkin is the set of rules that Cucumber uses. It is a “Business Readable, Domain Specific Language” that lets you describe software’s behavior without detailing how that behavior is implemented. It has 2 main purposes: Documentation and Automated Test creation.
To run properly our test cases using the definitions made in the .feature files, Cucumber has to receive a set of “Steps definitions” which map the Gherkin of each steps into code, that in this case is written in Java.
To develop is to test.
Yes, it makes sense. To run the automated test cases we use Selenium, that is a suite of tools where the most important are:
- WebDriver API
- Browser Driver (ChromeDriver in our case)
We use WebDriver API to develop our test cases. This API provides a lot of methods that we use for all the critical actions (from opening a browser instance to validating that one WebElement has the correct text).
The Browser Drivers are binaries that are responsible for handling the browser. An instance of a WebDriver needs to work with this to be able to translate the code into instructions that can be executed in the browser.
Selenium isn’t as easy to understand, but we think that the exampled detailed here is a very good one. It uses an analogy with a Taxi Driver to explain how it works and how it handles the requests made by the automated test cases.
Selenium provides a way to distribute the test across physical or virtual machines. It is called Selenium Grid and works great running test cases in parallel, speeding up the execution time and giving feedback. With the Grid we can take advantage of the infrastructure, as it brings the possibility to run multiple tests in parallel on multiple nodes, in a heterogeneous environment where we can have a mixture of different OS and browsers.
Using these technologies we created our Automation Framework.
An automation framework is an implementation of different technologies that, interacting together, create a system that allows to perform automation test. It has a lot of different features, for example:
- Modular
- Maintainable
- Easy to implement reusable test
- Parallel Execution
- Reporting
- Integration with CI
- Use of different environments
- Use of different devices
- Configurable
One critical point is the maintainability of our code. To ensure this maintainability we use a very popular pattern called Page Object Model.
POM is a design pattern that allows to have a better test maintenance and reduce the duplicated code.
The benefit is that if the UI changes for the page the test don’t need to change, only the code on the page object needs to be updated. There is a clean separation between test code and page specific code such as locators.
The PO could represent a part/components of an entire page.
Some useful rules to create a maintainable test code:
- PO should not make assertions, the assertions are part of the test.
- The PO should contain the representation of the page and the services the page provides with the correspondent method but not code related to what is being tested.
This image represents our test execution flow. It shows how the mentioned technologies work together:
This project is still growing. Our plans are to start with the mobile automation in a few months and we will create another article to show you what we learn with that. Stay tuned!
