Top 5 Most Rated Node.js Frameworks for End-to-End Web Testing

Hello my friends, today we are going to look into the most popular solutions for functional web testing. For my review, I listed the most well-known and popular frameworks, sorted them by the number of stars in their GitHub repos and picked top 5. Here are they (the number of stars is specified for the moment when this article was written, and it can differ from the current score).

• CasperJS — 6399 stars Site GitHub
• Protractor — 6351 stars Site GitHub
• Nightwatch.js — 5975 stars Site GitHub
• TestCafe — 2106 stars Site GitHub
• CodeceptJS — 1040 stars Site GitHub

CasperJS is written in Python, i.e. it is not a native Node.js solution. However, I’ve added this framework to my review because it can be installed from npm and so it fits well into the Node.js toolchain.

Further, we will have a detailed look at each of them. We will discuss their main features and try to perform a couple of basic actions with each, which will allow us to understand what each framework is worth. I’ll describe what you need to do to write your first simple test. This review does not encompass all the features — only the first impression from getting started with the framework. As a test scenario, we will use each framework to find its repository on GitHub.

CasperJS

This is the most starred framework in our rating. A special thing about this framework is that it’s tailored to be used with headless browser engines (PhantomJS or SlimerJS). On one hand, your tests run much faster compared to real browsers. And you don’t need to install browsers in your continuous integration system. On the other hand, the behavior of headless browsers sometimes differs from real browsers, which makes it impossible to detect browser-specific issues. Nevertheless, these considerations may not affect your choice of framework because you can run tests in PhantomJS with many other frameworks by using plugins or other simple mechanisms. Another advantage of CasperJS is support for CoffeeScript syntax in test code.

Installation

• Install PhantomJS npm install phantomjs -g
• Install Python 2.6 or newer
• Install CasperJS: npm install -g casperjs
• Create a file with the test: /casperjs/test.js

Test Code

This test code in plain JavaScript looks a little bit overburdened. It could get better if we used CoffeeScript, but after I rewrote the test in this language, I couldn’t get it to work because the version of PhantomJS should be older than 2. By looking at test code, you can also see that using arrow functions instead of nested plain functions, i.e. ES6 support, could improve code readability.

Run Test

casperjs test test.js

Report

• Test Passed

• Test Failed

Protractor

A very popular solution positioned as a framework for testing Angular and AngularJs applications. This allows it to attract many Angular users. Protractor has built-in means to work with Angular elements. This simplifies code significantly and allows you to write tests easily and fast. Though, it’s worth noticing that the same can be achieved in other frameworks one way or another. However, if you use Angular heavily, consider using Protractor. In contrast to the previous framework, Protractor tests against real browsers, which is an important advantage. Also, note the automatic waiting feature that, according to the authors, facilitates writing and running tests.

Installation

• Install Java Development Kit
• Install Protractor: npm install –g protractor
• Initialize Web DriverManager: webdriver-manager update
• Create a test file (/protractor/test.js)
• Create a configuration file (/protractor/conf.js)

Test Code

Run Test

• Run Web DriverManager: webdriver-manager start
• protractor conf.js

Report

• Test Passed

• Test Failed

Nightwatch

A very popular framework, the main competitor to Protractor. Its benefits include highly transparent and readable test code. Nevertheless, you still need to use waits in code while Protractor and TestCafe have built-in waiting mechanism. The main drawback is the most complicated environment configuration among all the reviewed frameworks. You can see for yourself how many steps it takes to run a simple test. In general, Nightwatch and Protractor have a lot in common.

Installation

• Install Java Development Kit (JDK). The minimum required version is 7
• Download Selenium — selenium-server-standalone-{VERSION}.jar
• Put Selenium in the same directory with tests
• Download the WebDriver for Google Chrome and put it in the same folder (/nightwatch/chromedriver.exe)
• Install the framework: npm install -g nightwatch
• Create a configuration file (/nightwatch/conf.js). In this file, you need to specify the path to the Chrome WebDriver.

• Create a test file (/nightwatch/test.js)

Test Code

Run Test

• Run Selenium java -jar selenium-server-standalone-3.0.0.jar
• nightwatch test.js

Report

• Test Passed

• Test Failed

TestCafe

A young but already popular framework. Its key feature is that it doesn’t use WebDriver to work with browsers as other solutions do. That is why it requires minimal test environment and it is installed with one command. Besides, this approach allows you to run tests on any physical device without requiring anything but the browser. Other pluses include writing tests in ES6/ES7, automatic waiting as well as the elegant and informative console reporter. The main drawback of TestCafe is that it emerged not long ago and has a much smaller community than its competitors. Nevertheless, the community grows bigger and TestCafe already has a wide range of plugins that extend its functionality. It is undoubtedly worth your attention.

Installation

• npm install testcafe -g
• Create a test file (/testcafe/test.js)

Test Code

Run Test

testcafe chrome test.js

Report

• Test Passed

• Test Failed

CodeceptJs

This is not a plain framework but a wrapper that provides syntax to write tests that will run in one of the popular test runners. You can choose from the following options

• WebDriverIO
• Protractor
• Selenium WebDriver JS
• NightmareJS
• or others…

The syntax provides a simple and comprehensible linear way to define the test scenario. You can extend commands in this syntax as you wish, though the set of built-in commands is enough to write basic tests. The support for ES6 syntax makes test code concise and readable. An advantage of such an approach is that you can run your test without substantial changes in any of the popular test runners with no additional effort.

Installation

• Install framework npm install -g codeceptjs
• Generate codecept.json config with a console command codeceptjs init
• Create a test by using the command codeceptjs gt
• … In addition, you need to install everything required to run the helper of your choice (e.g., Protractor)

Test Code

Run test

codeceptjs run --steps

Report

• Test Passed

• Test Failed