Fancy Assertion Libraries

A Response to “5 Questions Every Unit Test Must Answer” by Eric Elliot

If you’ve not read Eric Elliot’s excellent article about unit testing, go read it now. You’ll be (mostly) better off for it.

The article mentions a point I stress to anyone who is willing to listen, and probably to a few who aren’t…

A failing test should read
like a high-quality bug report.

This is a great point, and one which I wholeheartedly agree with. However, later in the article…

All those fancy assertion libraries with hundreds of different fancy assertions are destroying the quality of your tests.

And then later…

If the only available assertion in every test suite was `equal()`, almost every test suite in the world would be better for it.

Hang on a moment…

…those fancy assertion libraries are creating that great bug report we all want. I don’t believe you can get a great bug report, or for that matter a great readable test, just by using toEqual. At least, not always.

Let’s take an example

Let’s start with the example in the article. It tests that the compose() function returns a function. I’ve copied this test from the article.

And what happens when it fails — the bug report?

This isn’t a bad report, and if this were reported as a bug, it certainly contains the most important information needed to start to investigate the issue.

But let’s just take a look at that same test using a fancy assertion library

I’ve switched to mocha here, but that doesn’t really have an impact. I’ve also put the expected value directly in the assertion — if you wanted to put this in a variable called expected, then be my guest, but I don’t think it makes the test more readable by doing that.

I find that reading the assertion reads left to right in English helps with the clarity of the test, but that’s subjective.

assert.equal(actual, expected, 'compose() should return a function')

vs

expect(resultOfCompose, 'to be a', 'function')

Let’s see what happens when this test fails:

It’s given us the same information, and more — it’s shown us the actual object, and that it expected it to be a function.

I’d argue that this is a significantly better bug report. You can even see what the issue is, and can almost certainly instantly suggest two possible solutions to the problem.

Not to belittle toEqual

I like toEqual (or ‘to equal’), and the advice to use it wherever possible is certainly not bad advice. However, using it over and above everything else isn’t going to give you the best tests. Or the best failures.

And then there’s objects…

A question I stress whenever I’m writing or reviewing a test, is “what am I testing?”. A question that was one of the main topics of the article. (Actually, I’d like that to be the main takeaway from the article!)

This question can be used to check the test is checking the right thing at all sorts of levels.

Let’s say we have a search function, that searches some space for an object. If it doesn’t find it, it returns the closest match.

OK, so we’re asserting that the result object is correct.

“What am I testing?”

It’s actually testing a lot of things here. It’s testing that found property is true, that the match property is an object, the match id value is 42, and the match name is ‘steve’. That’s a lot for one test.

The test description says “finds a single value in the collection”, so we should probably just be testing the found property (the match property should be tested in a separate test). If the name doesn’t match, the test is going to fail, except, it did ‘find a single value in the collection’.

Fail for one reason

One key aspect to TDD, is to make sure that the tests fail for one reason, and one reason alone. There’s lots of reasons this test could fail, and many of them aren’t related to what we’re testing.

Let’s fix it

We can just test the found property, right?

Back to that bug report…

If we just check found is true, the failure report is going to say

expected false to be true

If you report a bug like that, don’t be surprised if it’s closed without comment :)

Let’s fix it, no, really, for real this time

Here I’m using ‘to satisfy’, a very powerful assertion from unexpected, that only checks the properties that are provided, and ignores the rest. This is normally my go-to assertion, it’s an easy enabler for testing only the things we’re interested in.

Now we’re only testing what we should be testing in this test, but if it fails, we see the following output

It shows us the full result, so we can see what it did find, and it shows us an annotated diff, highlighting where the issue is.

The Big Wide World

These are just simple examples, but in the real world there are result objects that can be large and complex. An obvious example would be a ReactElement, when testing components created with the React library. Testing single object properties here would be very long winded and practically unreadable. toEqual would be near impossible to use. Libraries such as expect-jsx and unexpected-react have emerged, to help with making the output readable.

Using unexpected-react, you get the detailed bug report, with a highlighted diff of the issue(s), diffed at the element/JSX level, not just a string representation of the element (as in expect-jsx).

There are plugins for unexpected for many other libraries and types, e.g. Sinon, Streams, express, that provide custom assertions and failure output. You can actually go even further with the tests in the examples to improve the readability of the tests and the usefulness of the failure reports with further assertions from unexpected.

Just my 2 cents

This is obviously just my opinion, and if using just toEqual works for you, then great.

However, I’ll be sticking to my fancy assertion library.