An Overview of JavaScript Testing in 2021

This guide is intended to catch you up with the most important reasoning, terms, tools, and approaches to JavaScript testing for the year 2021.

It combines information from the best articles recently released (they are referenced at the bottom) and adds from my own experience at Welldone Software, where we have implemented different testing solutions for different products over many years.

It’s a little long but whoever reads and understands this guide, can safely assume they know the overall state of JavaScript testing in the web development community.

Just a couple of years ago website testings had very little documentation, were slow, hard to implement, not deterministic, and in general not too fun to work on.

Nowadays, the website testing field has stabilized. Several testing tools receive very positive feedback from their users, well-known testing best practices has been created, and testing became a more integral part of the job of many developers.

Today’s cutting edge website testing tools are fast, informative, and easy to work with. These tools would make your testing and developing experience much more enjoyable.

According to The State of JavaScript. Jest, the leading unit test framework that we would discuss later in details, has it’s satisfaction rates at 96%!

Looking forward, I forecast a large entrance of AI into the field of automated testing. Some such tools already exist, and actively improving the workflow and experience of thousands of developers.

  • If you see anything I missed or got wrong, leave a comment and I’ll address it right away.

Enjoy :)

Test Types

You can read about different types of tests in more depth here and here and here.

In general, the most important test types for a website are:

  • Unit Tests- Testing of individual units like functions or classes by supplying input and making sure the output is as expected:
  • Integration Tests- Testing processes across several units to achieve their goals, including their side effects:
  • End-to-end Tests (also known as e2e tests or functional tests)- Testing how scenarios function on the product itself, by controlling the browser or the website. These tests usually ignore the internal structure of the application entirety and look at them from the eyes of the user like on a black box.

Running Tests

  • Tests can run in the browser by creating an HTML page with the test libraries and test files included as JS scripts.

Out of the three, we suggest using the third method (Node.js + jsdom) when possible, because it’s much faster than the other two. The first and second methods may be more reliable because you’re using the the same exact software which will render your site in real life.

Test Tools Types

Test tools can be divided into the following functionalities. Some provide us with only one functionality, and some provide us with a combination.

To achieve the most flexible set functionality, it’s common to use a combination of several tools.

  1. Test launchers are used to launch your tests in the browser or Node.js with user config. (Karma, Jasmine, Jest, TestCafe, Cypress, webdriverio)

Let’s explain some of the terms mentioned above:

Test launchers

Launch a list of tests based on a configuration you provide (what browsers to run in, what babel plugins to use, how to format the output, etc).

Testing Structure

Refers to the organization of your tests. Nowadays, tests are usually organized in a BDD structure that supports behavior-driven development (BDD). It often looks like this:

Assertion Functions

Are used to make sure that tested variables contain the expected value. They usually look like one of these:

TIP: Here is a nice article about advanced Jasmine and Jest assertions.

Spies

Attach themselves to functions to provide us extra information about them. For example, how many times were they called, in which cases, by whom, what was passed to them in each call?

Spies are used in integration tests to make sure that the side effects of a process are as expected. For example, how many times was a calculation function like execute in this case called?

Stubbing or Dubbing

Replaces selected methods of existing modules with user-supplied functions in order to ensure expected behavior during the test.

To ensure user.isValid() will always return true during a test, where you test something completely different, you can use:

Mocks or Fakes

Used to fake certain modules or behaviors to test different parts of a processes.

Sinon can, for example, fake a server to ensure offline, fast and expected responses when testing a certain flow.

Snapshot Testing

Allows you to compare a data structure to what it was in older releases.

The following example, from the official Jest documentation, shows a snapshot test of a certain Link component.

It doesn’t actually renders and takes a picture of the component, but it saves its internal data in a separate file like this:

When the test runs, and the new snapshot differs from the last one, the developer is prompted to confirm that the change is intended.

Notice: Snapshots are usually made to compare component representation data but they can also compare other types of data, like redux stores states or the inner structure of different units in the application.

Browsers Controllers

There are several ways to control browsers to simulate user behavior, like clicking the mouse, drag and dropping, typing, and navigating.

  • Some tools install drivers that are installed on top of them and control the browser using different methods. This is how selenium works:

Your code on Node.js <> WebDriver <> FF/Chrome/Safari Driver <> Browser

  • Another way would be to inject scripts of a JS code that has access to the whole application environment: DOM, network, cookies etc… to raise events that simulate user behavior. For example, here is how you simulate a click:

document.getElementByID('someButton').dispatchEvent(clickEvent)

  • Lately, a new way became very popular. Browser maintainers, exposed their own native APIs that lets you control the browsers directly. For example puppeteer:

Your code on Node.js <> Browser's API

Putting it All Together

To start testing you need to choose a testing structure that suits you, choose the style of assertion functions you like, and decide how do you want to run the tests.

We will discuss the tools you can choose from later in the article.

Some frameworks like Jest, Jasmine, TestCafe, and Cypress provide all of these out of the box. Some of them provide only some of the functionality and a combination of libraries can be used. A famous combinations of tools would be: mocha + chai + sinon.

We also suggest creating two different processes. One for running unit and integration tests and another one for Functional Tests. This is because functional tests usually take longer, especially when running the test suite on several different browsers.

Unit and integration tests can run on the fly, as you code, by using a “watch mode”. E2E tests, usually need much more time and are usually launched before merges and releases.

Unit Tests

Should cover all the small pure units of an application- utils, services and helpers. Provide all these units with simple and edge case inputs and make sure their outputs are as expected using the assertion functions.
Also make sure to use a code coverage reporting tool to know which units are covered.

Unit tests are one of the reasons to use functional programming and pure functions as much as possible.

The purer your application is, the easier you can test it.

Integration Tests

Old school tests were focused on unit testing and resulted in applications where many small parts were working but the processes as a whole kept on failing.
Integration tests, on the other hand, detect cases where a unit is refactored and passes its tests but a process that depends on it fails.

The best demonstration of why testing more than each part of a system separately is important can be seen in this great GIF.

I created this GIF based on a slide in this great lecture by
Viv Richards that I recommend to watch.

It’s also important to remember that in the real world, for the reasons of imperfect design and the widespread use of black boxes, not all units are pure and not all units are testable- some units can be tested only as part of a bigger process.

Integration tests should cover important cross-module processes. Sometimes they extend to processes across several classes and sometimes to testing different systems like Front-End-Back-End interactions.

Comparing to unit tests, you would probably benefit from using spies to ensure expected side effects instead of just asserting the output and stubs to mock and modify the parts of the process that are not in the specific test.

Component snapshot tests fall into this category as well. They provide us with a way to test how processes affect selected component structures and data structure without actually rendering them into a browser or browser-like environment.

Functional Tests

Sometimes the quick and effective unit and integration tests are not enough.

Functional tests control browsers and simulate user behavior on these environments (clicking, typing, scrolling etc…) and make sure these scenarios actually work from the point of view of an end user.

Also it is worth noting that many services provide you with devices and browsers to run these tests on. Here are even more services like these.

Visual regression testing tools can also be set up to verify that the different screens of your applications are ok visually by a smart comparison of screenshots. These screenshots are usually taken as part of your Functional Tests or by executing a separate session of browser automation.

Applitools smart visual regression tools

List of General Prominent Testing Tools

There are dozens of great tools out there. I couldn’t include all of them here but I tried to include the most important to know, best maintained , and the most adopted tools in the following list:

jsdom

jsdom is a JavaScript implementation of the WHATWG DOM and HTML standards. In other words, jsdom simulates a browser’s environment without running anything but plain JS.

As mentioned before, in this simulated browser environment, tests would run really fast. The drawback of jsdom is that not everything can be simulated outside a real browser (you can’t take a screenshot for example) so using it will limit your test’s reach.

It’s worth mentioning that the JS community rapidly improves jsdom and the current version is very close to support whatever exists on a real browser.

Storybook

While not, strictly speaking, a testing tool, Storybook lets you write components in special “stories” which enable developing and interacting with components in isolation.

Not only it encourages you to write your components in a more “testable” fashion, you can actually test the components in storybook, using Chromatic that we would discuss below.

Testing Library

Simple and complete testing utilities that encourages good testing practices. It is developed by Kent C. Dodds which is testing guru. I suggest to have a look at his work to learn more about best practices in testing websites.

The library provides special tools for different frameworks like React, Preact, React Native, Marko, Angular, Vue, and Svelte. The most famous of them is React Testing Library which is very widely adopted.

It also helps with E2E testing tools like Cypress, Puppeteer, Testcafe, and Nightwatch that would all be discussed next.

It helps with convenient selectors, firing events, configuration, dealing with asynchronous code, and many more.

Electron

The Electron framework lets you write cross-platform desktop applications using JavaScript, HTML and CSS. It also has a headless mode.

It has a huge community and many important applications are built on top of it, so it is supposed to stay up to date:
Atom, Slack, Skype, GitHub Desktop and many more.

Testing tools like Cypress.io use Electron to launch tests with maximum control of the browser.

Istanbul

Istanbul will tell you how much of your code is covered with unit tests. It will report on statement, line, function and branch coverage in percentages so you will understand better what is left to cover.

Karma

Karma hosts a test server with a special web page to run your tests in the page’s environment. This page can be run across many browsers and browser-like environments including jsdom.

Chai

Chai is the most popular assertion library. It has many plugins and extensions.

Unexpected

Unexpected is an assertion library with a slightly different syntax from Chai. It is also extensible so assertions can be more advanced with libraries that are based on it like unexpected-react that you can read about more in depth here.

Sinon.JS

Sinon has very powerful standalone test spies, stubs and mocks for JavaScript that works with any unit testing framework.

testdouble.js

testdouble is a less popular library that does what Sinon does, and claims to do it better, with a few differences in design, philosophy, and features that could make it useful in many cases. You can read about it here, here and here.

Wallaby

Wallaby is another tool worth mentioning. It is not free, but many users recommend buying it. It runs on your IDE (it supports all major ones) and runs tests that are relevant to your code changes and indicates if anything fails in real time alongside your code.

Cucumber

Cucumber help with writing tests in BDD by dividing them between the acceptance criteria files using the Gherkin syntax and the tests that correspond to them.

Tests can be written in a variety of languages that are supported by the framework, including JS, which we are focusing on:

Many teams will find this syntax more convenient than TDD.

Choose Your Unit and Integration Tests Framework

The first choice you should probably make is which framework you want to use.

* you can’t go wrong with Jest. It has nice approval ratings, it’s very fast, clear and has many features in case you need to cover complex scenarios. If you want to “just get started”, go with Jest.

* If you want a very flexible and extendable configuration, go with Mocha.

* If you are looking for simplicity go with Ava.

* If you want to be really low-level, go with tape.

Here is a list of the most prominent tools with some of their characteristics:

Jest

Jest is the testing framework created and maintained by Facebook. It spiked in popularity and became the most popular library throughout 2017 (Judging by State of Javascript 2017 vs 2018)

It is based on Jasmine which we will discuss later. Over time, Facebook replaced most of its functionality and added a lot of features on top of it.

  • Performance- First of all Jest is considered to be faster for big projects with many test files by implementing a clever parallel testing mechanism.
  • Snapshot testing- jest-snapshot is developed and maintained by Facebook, although it can be used in almost any other framework as part of the framework’s integration of the tool or by using the right plugins.

jasmine

Jasmine is the testing framework that Jest is based on. Why would you prefer Jasmine over Jest? It has been around for longer and it has a huge amount of articles, tools, and questions answered in various forums that were all created by the community over many years.

Also, Angular still suggests using it over Jest, although Jest is perfectly suitable to run Angular tests as well, and many people do it.

  • Ready-To-Go- Comes with everything you need to start testing.

mocha

Mocha is the most used library. Unlike Jasmine, it is used with third party assertions, mocking, and spying tools (usually Sinon and Chai).

This means Mocha is a little harder to set up and divided into more libraries but it is more flexible and open to extensions.

For example, if you want special assertion logic, you can fork Chai and replace only Chai with your own assertion library. This can also be done in Jasmine but in Mocha this change will be more clear and obvious.

  • Community- Has many plugins and extension to test unique scenarios.

AVA

Ava is a minimalistic testing library that runs tests in parallel.

  • Ready-To-Go- Comes with everything you need to start testing (besides spying and dubbing that you can add easily). Uses the following syntax for test structure and assertions, and runs in Node.js:
  • Globals- As you can see above, it does not create any test globals, so you have more control over your tests.

tape

Tape is the simplest of them all. It’s just a JS file you run with node with a very short and “to-the-point” API.

  • Simplicity- Minimalistic structure and assertions without a complex API. Even more than Ava.

Functional Testing Tools

First of all, as mentioned above, here and here you can find great articles about service providers that will host the machines where tests run and help you run these tests on different devices and browsers.

The tools for the purpose of functional testing differ very much from each other in their implementation, philosophy, and API, so it is strongly suggested to invest time in understanding the different solutions and testing them on your product.

* In short, if you want to “just get started” with a simple to set-up cross-browser all-in-one tool, go with TestCafe.

* For a convenient UI, clear documentation, cool tools and overall fun all-in-one tool Functional Testing experience go with Cypress.io.

* If you prefer older and more time-proven tools, you can “just get started” with Nightwatch.js.

* If you prefer older and more time-proven tools, with the maximum community support and flexability, WebdriverIO is the way to go.

* If you want the most reliable and Angular friendly solution, use Protractor.

selenium

Selenium and tools that rely on it dominated the market of Functional Tests for years. It is not written specifically for testing and can control a browser for many purposes by exposing a driver that controls browsers using add-ins and browser extensions.

Node.js <=> WebDriver <=> FF/Chrome/IE/Safari drivers <=> browser

Selenium WebDriver can be accessed in many different ways and using a variety of programming languages, and with some tools even without any real programming.

The WebDriver can be imported into your testing framework and tests can be written as part of it:

The WebDriver itself might be sufficient for you and indeed some people suggest using it as is, but various libraries were created to extend it either by forking and altering it or by wrapping it.

And indeed wrapping the WebDriver might add redundant code and could make debugging harder, whereas forking it might diverge it from the very active ongoing development of the WebDriver.

Still, some people prefer to not use it directly. Let’s look at some of the libraries for selenium:

Protractor

Protractor is a library that wraps Selenium and provides us with improved syntax and special built-in hooks for Angular.

  • Angular- Has special hooks, although it can successfully be used with other JS frameworks too. Angular official documentation suggests using this tool.

WebdriverIO

WebdriverIO has its own implementation of the selenium WebDriver.

  • Syntax- very easy and readable.

Nightwatch

Nightwatch has its own implementation of the selenium WebDriver. And provides its own testing framework with a test server, assertions, and tools.

  • Framework- Can be used with other frameworks too, but can be especially useful in case you want to run functional tests not as part of another framework.

Appium

Apium provides an API similar to Selenium for testing websites on a mobile device using the following tools:

So if you use Selenium or Selenium based tools, you can also use Apium to test on mobile devices.

TestCafe

TestCafe is a great alternative to Selenium-Based tools. It was rewritten and open-sourced at the end of 2016.

TestCafe had also a paid version that offered non-programming testing tools. It has been deprecated and got replaced by the new TestCafe Studio.

TestCafe injects itself into the website as JavaScript scripts instead of controlling the browsers themselves like Selenium does. This allows it to run on any browser, including on mobile devices, and have full control over the JavaScript execution loop.

Cypress

Cypress is a direct competitor of TestCafe. They are doing relatively the same, which is injecting tests into a website, but they try to do it in a more modern, flexible and convenient way.

The difference between them is that Cypress.io runs itself in the browser and controls your tests from there where TestCafe runs in Node.js and controls the tests through a serialized communication with its injected script in the browser.

The library is relatively new (moved from closed beta to public beta at October 2017) but they already have many adopters and enthusiasts.

  • Parallel testing was introduced in version 3.10.

Puppeteer

Puppeteer is a Node.js library, developed by Google. It provides a convenient Node.js API to control Chrome or Headless Chrome.

Headless Chrome is just a regular Chrome v59+ that is launched with the
--headless flag. When Chrome is run in headless mode, it exposes an API to control it, and as said before, Puppeteer is the JavaScript tool that Google provides to control it.

Here it is worth mentioning that Firefox has also released their headless mode at the end of 2017.

Notice that different testing tools can also use Headless Chrome and Firefox. For example: TestCafe, Karma, Cypress.

  • Puppeteer is maintained by google and has a big community that uses and develops tools and wrappers around it.

Playwright

Playwright is a exactly like Puppeteer, but it is developed by Microsoft (launched at January 2020 by a team that originally developed Puppeteer) and strives to be cross-platform. It also has a slightly different API and new features like waiting for elements to be available before clicking on them.

It’s worth mentioning that since the library is pretty fresh, breaking changes may be made in the near future and some things might not work or be documented as expected at the moment.

Here is a nice comparison of the library with Pupetter, and here is an article about it with an example of how to use it.

PhantomJS

Phantom implements the chromium engine to create a controllable Chrome-like headless browser. It was a great tool to run in headless mode until Google announcement of “Puppeteer”.

It’s main maintainer, Vitaliy Slobodin, no longer works on it, and its development was suspended and its repository archived.

Nightmare

Nightmare is a Functional Testing library that offers a very simple test syntax.

It uses Electron which uses Chromium to control the browser’s behavior.

Doesn’t seem to be maintained recently. Probably because of the introduction of “Puppeteer” as well which provides you with the same features out of the box.

CodeceptJS

Like CucumberJS which was discussed above, Codecept provides another abstraction over different libraries’ API’s to make your interactions with tests use a slightly different philosophy that focuses on user behavior.

Here is how it looks:

And here is the list of libraries that can be executed using this code. All discussed above.

WebDriverIO, Protractor, Nightmare, Appium, Puppeteer, and even the new Playwrigh.

If you believe this syntax is better for your needs, give it a shot.

Visual Regression Testing

The visual regression testing tools are consists roughly of the following:

  • Techniques and integrations to automate the browser or to run as part of Functional Testing tools discussed above, including in the CLI for CI.

There are a lot of tools of this type in the market, but it feels like this field still has a long way to do.

Also I noticed that paid tools in the visual regression testing category are much better than the free ones.

Applitools

  • Easy to set up.

Percy

  • Easy to set up.

Happo

Happo is a paid visual regression testing tool. It hooks into the CI to compare the visual appearance of UI components before and after a change.

Screenshots can be taken in different browsers and across different screen sizes to ensure consistent cross-browser and responsive styling of your application.

Paid with a free plan for open source projects.

LooksSame

Yandex created this library alongside with the now deprecated Gemini, that was a great simple-to-use visual regression testing tool.

Yandex now migrated to hermione that runs tests using WebdriverIO v4 and Mocha.js and uses LooksSame for visual regrations. It is more simple and limited than the paid tools mentioned above, but for simple websites it can be enough.

LooksSame can also be used on it’s own as long as you generate screenshots in any way you like.

BackstopJS

An open source visual regression utility that runs on Chrome Headless with Puppeteer and CI support.

AyeSpy

An open source utility by the Times Tooling team at News UK.

Uses selenium docker to create visual regression tests on Chrome / Firefox.

reg-suit

An open source library that compares images, generates reports and saves them on the cloud. Very convenient if you want to add visual regression tests to an existing functional test. Simply add steps of taking screenshots to your existing test flow and use it to compare these screenshots.

Differencify

Another open source Chrome Headless using Puppeteer testing tool with nice integrations with Jest snapshots. Can run in docker and generates convenient reports.

No Coding Functional Testing Tools

testim

Opens your application in a separate window and uses a browser extension to record your manual interactions with the application as test scenarios.

Uses machine learning to help you record and validate test scenarios. Cross Browser and has some nice integrations with many CI and Collaboration tools.

Has free and paid flexible plans.

Chromatic

https://www.chromaticqa.com/

Chromatic is a visual testing tool for Storybook made by Storybook maintainers.

Diff Detector renders the UI of each story and takes a visual snapshot. If anything changes, you are prompted to accept the visual changes locally and as part of the project’s pipelines.

Screener

Lets you record your tests using a chrome extension, has in depth visual regression reporting. Has some nice integrations like with storybook different CI tools and BrowserStack and Sauce Labs.

Not free.

Other tools of this type (you are free to suggest more in the comments):

Conclusion

We reviewed the most trending testing strategies and tools in the web development community and hopefully made it easier for you to test your sites.

In the end, the best decisions regarding application architecture today are made by understanding general patterns that are trending in the very active community of developers, and combining them with your own experience and the characteristics of your application.

Oh, and writing, and rewriting, and rewriting, and rewriting, and testing different solutions :)

Happy testings :)

Thanks :)

Suggested Links

Older versions of this article:

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store