Javascript testing for the 21st century

Tests are absolutely a must. Without tests, it is not possible to evolve software.

Yet, tests are additional code to write and maintain.

And they break; which is great if they do to inform us that we were about to introduce a defect in our API; frustrating (to use a euphemism) otherwise.

Software testing is defined as the process of determining the correctness of software under the assumption of some specific hypotheses.

But that definition on its own does not tell an important truth

A truth so important that all other things should be derived from this one. And all other things won’t matter if this truth is does not hold.

It goes like this:

If a test fails it must tell you why.

If the above statement does not hold true , it does not matter how good all other parts are.

• It does not matter that the tests are fully integrated in a continuous build environment.
• It does not matter that the whole suite of tests executes under a few minutes.
• It does not matter about the test coverage being close to 100%.

Please, don’t get me wrong. All parts are important; but the ability to report failure in the most accurate way beats all others.

So how do we get there ?

• Simplify
  You must move towards a minimalistic approach. Ideally you should not rely on any additional library except the one you are testing.
• Write sequential.
  Some of the best practice to write programming code do not apply for testing code. Testing code should be flat and sequential, should avoid cyclomatic complexity (except for test-checks). It should also avoid function definition and any form of parametrization.
• Make it a program
  Let your test be a program. If it runs successfully then your tests have passed, if it fails looking at the program and the line/lines where it failed should tell you why.

yolpo

It’s always been common to use code snippets in technical documentation to better illustrate the functionality of any given API.

We know that if a picture might be worth a thousand words, so is a code snippet to developers.

Consider the following snippets taken from http://lodash.com

_.last([1,2,3]); // → 3
_.assign({ 'a': 1 }, { 'b': 2 }, { 'c': 3 }); 
                            // → { 'a': 1, 'b': 2, 'c': 3 }
_.map([1, 2, 3], function(n) { return n * 3; }); 
                            // → [3, 6, 9]

We can make a test of them

x = _.last([1,2,3])
if (x !==3 ) throw '';
x = _.assign({ 'a': 1 }, { 'b': 2 }, { 'c': 3 });
if (x == undefined) throw '';
if (x.a != 1) throw '';
if (x.b != 2) throw '';
if (x.c != 3) throw '';
x = _.map([1, 2, 3], function(n) { return n * 3; });
if (x == undefined) throw '';
if (x.length !== 3) throw '';
if (x[0] != 3) throw '';
if (x[1] != 6) throw '';
if (x[2] != 9) throw '';

Arguably, we have traded some of the beauty of using comments to express the expected results, but now we have made a regression test for it.

We still need one final thing: a bit of code to identify uniquely the library under test:

load('https://cdnjs.cloudflare.com/ajax/libs/lodash.js/3.10.0/lodash.min.js');

The load function might require a bit of explanation. Yolpo introduces the global function load because javascript does not have a mechanism to load a script from a URL synchronously.

So now let’s take a look at the complete code:

There are a few possible outcomes.

• It runs successfully.
  Great news. Our regression test (no matter its coverage) has passed.
• It fails at line 5.
  We would then ask yourselves. Have I lost the internet connection ? Is cloudfare down ?
• It fails at any line 7–19.
  Fair enough, but now for every line we should be able to work out which function in lodash is broken and how.

See the running example here:
http://www.yolpo.com/embed.html?gist=933277296df75a382a16

Summary

You do not need any framework/testing library to test. Your testing code should be simple, sequential javascript.