Let’s Compare Javascript Testing Frameworks

Testing your code is one of the most important things that you can do to make yourself a better coder. That’s not just because testing can prevent you from breaking previously-working code when you make changes (also known as regressions), but more importantly, it forces you to write better code in the first place.

We have known about the benefits of testing to prevent regression for a long time. That testing encourages better code in the first place is a viewpoint that is honored more in theory than practice.

In this article, I will compare javascript testing frameworks, so that you can see how easy it is to get started, and see which one makes the most sense for you.

Samples of Jest and Mocha test runs

The Test Frameworks

In order to demonstrate various Javascript testing frameworks, I made a simple Javascript module, Math.js, and I wrote tests for it in the following frameworks.

1. Ava
2. Jasmine
3. Jest
4. Mocha
5. Tape

Mocha and Jasmine are the two most popular testing frameworks right now. Jest, created by Facebook, is a wrapper around Jasmine, with auto-mocking, to make testing React easier. Tape, and now Ava take a somewhat more traditional approach to testing; they remind me of testing in Perl and Ruby.

The System Under Test

The sample module, Math.js, is going to be really simple. It has four functions, each of which take two arguments and return a result. You can probably guess what these functions do.

• Math.add()
• Math.subtract()
• Math.mulitply()
• Math.divide()

I’ll present the tests as if we are doing Test-Driven Development (TDD), so you can get a feel for the process. If you want to see the complete Math module, along with tests for each function in all five of these frameworks, you can download the complete example repo, testing-javascript-testing-frameworks, from GitHub.

If you want to follow along, run the following commands in a terminal to get started.

mkdir testing-javascript-testing-frameworks
cd testing-javascript-testing-frameworks
npm init -y
npm install --save-dev ava jasmine jest-cli mocha tape

Testing with Ava

First up, let’s look at an Ava test file. By default, Ava looks for files in a test/ directory; so, we’ll create a directory with that name. It’s also good practice to name your test file after the feature you are testing, but with “.test” between the filename and extension.

File: test/Math.test.js
const test = require('ava');
const math = require('../Math/Math');
const firstOperand = 2;
const secondOperand = 3;
test("Math add function", t => {
  const result = math.add(firstOperand, secondOperand);
  t.is(result, firstOperand + secondOperand);
});

In the first line, we import the Ava module’s default export, its test function, and save it in the variable test. Next, we load the module we want to test. Then, we define a couple of variables we will use repeatedly throughout the tests.

To create an actual test, we call the Ava test function, and pass it two parameters: a name for the test (optional), and a callback function which will be called by Ava when it runs the tests. Your callback function will, in turn, receive one parameter — an execution object. We will use this execution object to make assertions about the behavior of the thing we are testing.

In general, in a test you have to do two things: call a function, and assert something about the return value of that function. Here, we call the Math module’s add function with our two test parameters, and then compare the result to the sum of those two numbers that we get by using the more traditional method of using the plus (+) operator. For this first test, we are using the execution object’s is method, which tests for equality. (The complete list assertions available in Ava is in the documentation.)

(Two interesting things about Ava: One, as you may have noticed, we used ES6 syntax in our test file. Ava automatically transpiles ES6 code in tests files to ES5 code. It does not transpile the JS files you are testing. Second, Ava runs tests concurrently, which you won’t notice with just one or a few tests.)

If you’re following along, you can try running the test now.

ava

Unless you skipped ahead and defined a Math module, you should get an error that begins like this.

Error: Cannot find module '../Math/Math'

We have not written any of the Math module yet, so Node cannot find anything to import. To get past this first hurdle, create a Math directory and, inside it, create an empty Math module file. As long as Node can find a file with the name of your import, it will create a empty object for it.

Now, when you run Ava, you should get this error.

X Math add function failed with "math.add is not a function"
1 test failed [08:21:01]
1. Math add function
  TypeError: math.add is not a function
    Test.fn (Math.test.js:8:23)

Finally, define an add() function in the Math module to make the test pass.

File: Math/Math.js
'use strict';
var Math = {
  add(a, b) {
    return a + b;
  }
}
module.exports = Math;

Run Ava one more time. The test should pass.

Math add function
1 test passed [08:28:18]

Eureka! We have created a Javascript test with Ava. Now, let’s see how we would make the same test in the other frameworks.

Testing with Jasmine

Here is the same test with Jasmine.

File: spec/Math.spec.js
var math = require('../Math/Math');
describe("Math", function() {
  var firstOperand;
  var secondOperand;
  beforeEach(function() {
    firstOperand = 2;
    secondOperand = 3;
  });
  it("should add two numbers", function() {
    var result = math.add(firstOperand, secondOperand);
    expect(result).toEqual(firstOperand + secondOperand);
  });
});

In contrast to Ava, you don’t need to import Jasmine, because it adds globals for all its functions. (This is one reason some prefer Ava or Tape to Jasmine or Mocha, because both the latter frameworks create globals by default.) You just import the Math module.

Jasmine allows you to group your tests inside a describe() function. You can even nest describe functions. Within a describe block, you run setup code before every test in the block by using a beforeEach() function. This allows you to create a clean environment for every test. There is also an afterEach() if you need to run teardown code. You can use multiple beforeEach and afterEach blocks, if you have multiple things you need to do and want to keep them organized.

In this case, we are just doing something trivial in the beforeEach block. We are setting the value of some function-scope variables that we will be using in our tests. Although it is not necessary here, because we are never modifying those values, this illustrates how you can use a beforeEach() block.

Unlike the other testing frameworks considered here, to use Jasmine with Node you need to create a configuration file. By convention, Jasmine tests are put in a spec/ directory, and Jasmine will look for its settings in a spec/support/ subdirectory. Put the following jasmine.json file in that directory.

File: spec/support/jasmine.json
{
  "spec_dir": "spec",
  "spec_files": [
    "**/*[sS]pec.js"
  ],
  "helpers": [],
  "stopSpecOnExpectationFailure": false,
  "random": false
}

After you create that settings file, just run the Jasmine command to run this test.

jasmine

You should see this output.

Started
.
1 spec, 0 failures
Finished in 0.005 seconds

Testing with Jest

Jest is actually a wrapper around Jasmine, but Jest also automatically mocks everything about your system, and you have to turn off that mocking for what you are going to test.

Unless you specify otherwise, Jest looks for tests in a __tests__ directory; so, let’s put one there.

File: __tests__/Math.spec.js
jest.unmock('../Math/Math'); // unmock to use the actual implementation of Math
var math = require('../Math/Math');
describe("Math", function() {
  var firstOperand;
  var secondOperand;
  beforeEach(function() {
    firstOperand = 2;
    secondOperand = 3;
  });
  it("should add two numbers", function() {
    var result = math.add(firstOperand, secondOperand);
    expect(result).toEqual(firstOperand + secondOperand);
  });

This test is exactly like the Jasmine test, but we also have to call jest.unmock() with the module that we actually don’t want to mock, so that we can test it.

We can run this test file with this command:

jest

and the output should look something like this.

Using Jest CLI v14.1.0, jasmine2
 PASS  __tests__/Math.spec.js (0.067s)
1 test passed (1 total in 1 test suite, run time 0.786s)

Testing with Mocha

Mocha is similar to Jasmine, but it is much more configurable. In fact, whereas Jasmine comes with built-in assertions and mocking, Mocha does not come with either. Instead, you have to explicitly add an assertion library, and a mocking library (if you need one).

We won’t cover mocking in this article, but here is how you would include Node’s built-in assert module to test in Mocha.

File: mocha-spec/Math.spec.js
var assert = require('assert');
var math = require('../Math/Math');
describe("Math", function() {
  var firstOperand;
  var secondOperand;
  beforeEach(function() {
    firstOperand = 2;
    secondOperand = 3;
  });
  it("should add two numbers", function() {
    var result = math.add(firstOperand, secondOperand);
    assert.equal(result, firstOperand + secondOperand);
  });
});

To keep this example simple, we are requiring Node’s built-in assert module in line 1, but there are many more robust assertion libraries that you can use with Mocha. Otherwise, this example looks a look like the Jasmine test.

Because we are demonstrating so many test frameworks together, I had to put the Mocha tests in a custom-named directory, to prevent other frameworks from trying to run them, and vice versa. Run the Mocha test like this.

mocha mocha-spec

and the output should look something like this.

Math
   should add two numbers
1 passing (7ms)

Testing with Tape

Tape, like Ava, uses a more traditional syntax for tests, and like Ava it does not add to the global namespace as Jasmine and Mocha do. Unlike Ava, Tape does not automatically transpile ES6 test code to ES5, nor does it run tests concurrently.

Here is the same test for Tape.

File: tape-test/Math.spec.js
var test = require('tape');
var math = require('../Math/Math');
var firstOperand = 2;
var secondOperand = 3;
test("Math add function", function(t) {
  var result = math.add(firstOperand, secondOperand);
  t.equal(result, firstOperand + secondOperand);
  t.end();
});

It’s worth pointing out one key difference between Tape and Ava. Tape needs to know when all the assertions for each test case are done. You can do this in one of two ways. Either you call the end() method on the execution object in your test callback, as we did above when we called t.end(), or you call t.plan(n), where n is the number of assertions that you plan to run in this test case. If you don’t do one or the other, your tests will hang.

I am putting the Tape test file in a custom directory, again to avoid conflicts with the other test frameworks. When you run this test file with

tape tape-test/*.js

then the output should look something like this.

TAP version 13
# Math add function
ok 1 should be equal
1..1
# tests 1
# pass  1

Next Steps

For practice, try to write tests for the subtract(), multiply(), and divide() functions for the Math module, and then write the functions to make your tests pass. You can pick one test framework, or try your hand at all of them.

For guidance, you can refer to my example repo on GitHub, testing-javascript-testing-frameworks.

Conclusion: Use whatever you like, as long as you write tests

From these examples, you can see the wide variety of styles in Javascript testing frameworks. Which one is best?

The best testing framework is the one that you like best, whether you like it because it’s easiest for you to understand, or because it fits in with the way you code, or for whatever other reason you like.

Once you’ve decided to try a testing framework you can use it to gain two important benefits.

1. As you make changes, add features or fix bugs, the tests can be run in a consistent, reliable, and even automated way. That way, you can be confident you did not break anything with your new code that was previously working.
2. You can use TDD (or its hipster cousin, BDD), to help you think more clearly about your design and the code you are writing.

To borrow an analogy from Kevin Ennis, testing is like wearing a seatbelt. It may not guarantee accidents won’t happen, but it can make the consequences a lot less severe. I will expand that analogy to answer another argument against testing, namely, that it is a waste of time. Think of all the time everyone spends throughout life putting on and taking off seatbelts. Yet, the overall benefits from using them far outweighs the time it takes.

So, go forth and test.

Originally published at Van Wilson’s Site.