The 10 Best Test Automation Tools
We all want to build maintainable tests for our web applications.
As a part of this goal, we want to avoid losing sleep over implementation details and focus on making our tests give us the confidence for which they are intended.
Those tests should be maintainable in the long run so regular changes don’t break them and slow our team down.
Iteration Speed / Realistic Environment
Some testing tools give you a short feedback loop between making a change and viewing the result, but don’t mimic the browser behavior precisely.
Other tools might use a real browser environment, but reduce the iteration speed and may be flakier in a continuous integration system.
1. Selenium
A well known framework for automating browsers and testing web applications.
The tests can be written in various programming languages, such as Java, C#, Ruby, JavaScript, R and Python.
Selenium currently provides client APIs for each of those languages.
Where possible, Selenium WebDriver uses native operating system level functionality rather than browser-based JavaScript commands to drive the browser.
This bypasses problems with subtle differences between native and JavaScript commands, including security restrictions.
It offers a lot of flexibility and it even supports dealing with iframes and multiple browser tabs.
The cross-browser capabilities are impressive, since Selenium tests can be executed on all major browsers (Chrome, Firefox, Safari, Edge, Internet Explorer).
Selenium Grid can be used with the WebDriver to execute tests on remote systems.
The only disadvantage of using Selenium is that it requires a considerable amount of skills and time to write tests.
For someone who doesn’t have programming experience, it might seem easy at first to write tests with Selenium, but not applying good practices will lead to an internal test automation framework which is difficult to maintain and is not reliable.
Website: https://www.selenium.dev/
2. Endtest
Endtest is best known as an Intelligent Test Automation solution.
It uses multiple open source and proprietary components to simplify the process of building and executing tests.
One of those components is a Recorder which empowers users without any programming skills to build automated tests.
You can edit and manage your tests without writing any code, through a GUI.
It has the same level of flexibility as Selenium, allowing you to automate scenarios which involve iframes, multiple browser tabs, file uploads, Shadow DOM, etc.
It has the same cross-browser capabilities as Selenium, the tests can be executed on all major browsers (Chrome, Firefox, Safari, Edge, Internet Explorer).
A cross-browser cloud grid is also included, which includes browsers on Windows, Mac machines and mobile devices.
Additional components include the capability to execute pieces of JavaScript code, to send API requests and to connect to databases and execute SQL queries.
These additional components can be used to add extra verification steps during your tests and to prepare or clean the test environments.
Website: https://endtest.io
3. Watir
An open source Ruby library for automating tests.
Watir interacts with a browser the same way people do: clicking elements and filling out inputs.
It actually uses Selenium under the hood, providing the same flexibility and cross-browser support.
It also supports iframes and multiple browser tabs.
The main advantage is the API is a bit easier to use, adding a layer of simplicity over the heavy Selenium API.
The main disadvantage of using Watir is that it’s less popular than Selenium and that makes finding answers for your questions a bit more difficult.
Website: http://watir.com/
4. Puppeteer
A Node library which provides an API to control Chromium or Chrome over the DevTools Protocol.
I see it as a potential replacement for Selenium in the distant future.
It has the flexibility to deal with scenarios that involve iframes and multiple browser tabs.
It’s worth knowing that there are 2 different packages: puppeteer-core and puppeteer.
The only difference is that puppeteer-core doesn’t automatically download Chromium when installed.
The main disadvantage is the lack of cross-browser capabilities, since it only works with Chromium-based browsers.
Even if Opera and Edge moved to Chromium, there’s no guarantee that Firefox and Safari will do the same in the future.
Another disadvantage is that the only supported language is Node.js.
Even if everything seems to revolve around JS in 2020, some users might want to use other languages.
Website: https://pptr.dev/
5. Playwright
A Node library to automate Chromium, Firefox and WebKit with a single API.
Playwright is built to automate a growing set of web browsers.
The experience is similar to Puppeteer.
It can automate scenarios that involve multiple page, domains and iframes.
Emulating mobile devices, geolocation and permissions is also possible.
Even if it has a wider cross-browser support, it does not support Internet Explorer.
Because it’s a relatively new library, you might not find a lot of resources online just yet.
If you have solid experience with writing Node.js code, this might be an interesting option to consider.
6. Sikuli
Sikuli can automate anything you see on the screen of your desktop computer running Windows, Mac or Linux.
It’s using image recognition powered by OpenCV to identify visual components.
This is handy in cases when there is no easy access to a GUI’s internals or the source code of the application or web page you want to act on.
It’s actually the technology behind most modern RPA solutions.
The advantage of using Sikuli is that it doesn’t matter if your elements have unique attributes or if those attributes change, because Sikuli relies only on visual recognition.
The main disadvantage is that the visual tolerance makes it difficult to automate across different browsers and screen sizes.
Programming skills are required to use this library and you can choose between Python, Ruby and Java.
If you have hybrid automated tests, which involve web applications and desktop applications, a good practice is to use Selenium for the web components and Sikuli for the desktop components.
Website: http://sikulix.com/
7. Micro Focus UFT
A proprietary solution, formerly known as QuickTest Professional (QTP).
It offers capabilities to automate web applications and desktop applications.
It’s using the Visual Basic scripting language to locate and interact with the elements from applications.
It also offers an IDE where you can build your test without writing any code, simply by creating a flow in a GUI.
This tool has been around for a long time and it did lose a lot of market share to Selenium.
Website: https://www.microfocus.com/en-us/products/uft-one/overview
8. IBM Rational Functional Tester
A proprietary tool that provides automated testing capabilities for functional, regression, GUI and data-driven testing.
It does include limited cross-browser capabilities, since it works only with Chrome, Firefox and Internet Explorer.
The main advantage is that it can also be used for automating desktop applications.
One of the advantages is that it has an Eclipse Java Developer Toolkit editor, which makes it easy for your team to code test scripts in Java with Eclipse.
The Toolkit editor includes features such as code completion and advanced debugging options.
Integrating IBM Rational Functional Tester with a CI/CD system might prove to be difficult, since it’s designed to be used on your own workstation.
Website: https://www.ibm.com/us-en/marketplace/rational-functional-tester
9. Jest
Jest is a JavaScript framework which allows you to access the DOM via jsdom.
It’s worth knowing that jsdom is only an approximation of how the browser works, it may be good enough for testing React components, but it is not a relevant option for reliable cross-browser testing.
Jest does provide a good iteration speed and it has powerful features like mocking modules and timers.
Ideally, it should be used by developers to test certain components, but the results will not guarantee that the application works as expected in the actual browsers.
Website: https://jestjs.io/
10. Cucumber
Cucumber is an open-source Behavior Driven Development (BDD) tool.
It supports multiple languages, including Ruby, Java, Scala and Groovy.
Building the test scripts does require programming skills.
The actual functions can be called by using Gherkin.
The obvious advantage of using Cucumber is that the scenarios can be easily understood by non-technical users.
It would be correct to say that Cucumber is not a replacement for Selenium, it’s simply a wrapper over Selenium.
A combination of Selenium and Cucumber increases the readability of the code and makes it more easy to re-use components.
Website: https://cucumber.io/
