Aggressive Elegance: rethinking testing with Distilled
Distilled is a young, unvetted library, hardly ready to be used in production development. If you have input, please provide it; I am eager and hungry for your perspective and suggestions.
Distilled is an aggressively elegant Javascript testing library, built to be highly extensible while remaining conceptually simple. At first glance it may seem weird that this library exists, because if you want to do unit testing in Javascript, you already have some options.
However even with a rich ecosystem already in place, Distilled was built out of necessity, and I want to take a moment to talk about what exactly that means.
Distilled abandons several assumptions that existing frameworks take for granted in order to solve deeper conceptual problems. But before getting into that, let’s talk about the current state of Javascript testing.
Testing frameworks are opinionated
This should not surprise anyone because unit testing is an opinionated subject, often for good reason. But it means that the barrier for entry in most testing frameworks is higher than it needs to be. By including both core concepts and meta-level best-practices into a single package, we make testing much less approachable for people without existing domain knowledge.
This is further complicated because not everyone agrees on what best-practices are. Use two different libraries and you’ll get different opinions about whether or not beforeEach is evil.
Testing frameworks do too much
To combat this, several libraries (in particular, Mocha) use large swaths of user configuration. In theory, this is a very good thing (tm). And I don’t necessarily want to discourage developers from adding options to their libraries.
However, configuration often masks more structural problems. It can provide a middle road when there are genuinely two separate paths to take, but at some point you should be asking yourself why flexibility only exists inside a JSON file. At some point we should be asking ourselves whether or not unit testing actually needs this level of setup.
Frameworks like Mocha (and many others) adopt a perspective that there are a finite number of setups most developers need. By covering each setup disparately, all needs can be met — never mind if those setups are incompatible with each other, or if they force developers to make decisions before they have the context to make them well.
Testing frameworks treat asynchronous as an addon
This may seem like a minor quibble, but it’s common especially for newer developers in the JS community to dance around and avoid callbacks or promises. Asynchronous code is good. It’s obviously good on the server side, but it’s also great on the browser side, because blocking the browser’s render loop is almost always a bad idea.
However, unit tests often feel antagonistic towards this style of code. The assumption has always been that the majority of tests would be written synchronously. As a result, the async mechanisms and tools within many popular testing libraries feel like hacks.
If I were to look at the above objections and pull out specific areas of testing I would like to improve, it would be:
- Lowering testing’s barrier of entry for new developers
- Encouraging existing developers to think more about test structure, rather than blindly following templates or preset rules
- Discouraging developers from adopting a one-size-fits-all mentality about their libraries
- And of course, that async thing
At first, some of these ideals seem mutually exclusive. If we want to make testing easy, we should do more for the user, right? Being less opinionated seems like it would take us in the opposite direction.
However, I stand behind building software APIs that are composable, meaning that low-level concepts are reduced until they are simple and elegant. These simplistic building blocks can then be used to build more complicated features and frameworks.
It is a revolutionary concept that I discovered entirely by myself without any outside help.
These are likely not new concepts for you.
However Distilled doesn’t just look towards elegance as an ideal, it treats it like a mantra. Inessential features are thrown away, vastly reducing the amount of conceptual overhead necessary to get started. But, because Distilled is extensible and unopinionated, these features can be added back and customized when they become necessary.
Let me give you some examples.
Assertions
suite.test('name', function () {
assert.ok(true, 'should be true');
});suite.test('name', function () {
if (!true) { throw 'should be true'; }
});
Assertions are syntactic sugar. In reality, code fails either because it can’t complete an operation or because an operation returns an unexpected result. Let’s get rid of them.
As a result, Distilled is not concerned with the debate between assertion style and BDD style tests, because as long as you’re throwing exceptions Distilled just works (tm). By throwing something away, we get to have our cake and eat it too — a flexible library that works with many assertion styles, but does not require them.
Test Runners
Rather than require you to read special documentation about how your code should be set up, rather than giving you a CI tool to call within NPM, Distilled asks developers to write their own test harnesses.
var Distilled = require('Distilled');/*
* Attach singleton since most testing libraries are singletons
* - but, your needs may vary.
*/
Distilled.suite = new Distilled();
require('./distilled/tests.js');
require('./distilled/callback.js');
//etc...
This seems antithetical to the idea of simplicity, but I want to posit that testing harnesses and node scripts are not nearly as difficult as we make them out to be. Unless built-in harnesses vastly reduce the amount of complexity, this type of rigid, magic behavior is more likely to do harm than good.
I posit that writing npm scripts is well within capabilities of even new developers. As a result, Distilled offers roughly the same level of complexity as many existing libraries, but an absurd level of customization, as well as the ability to evolve and change your harness as your needs change.
Synchronous test attachment
Let’s get weird. Virtually every testing library that I know of assumes that tests will either be synchronously defined, or attached as a separate step to being run.
Remember the stuff I was saying above about asynchronous testing? Distilled does not make this assumption.
suite.test('outer', function () {
this.test('inner', function () {});
});Distilled is asynchronous first in the most literal way possible. Distilled is a wrapper around native Promises, which means that every test has its own promise and that tests can be infinitely chained to other tests.
We have to give some stuff up to make this work — you can’t predict beforehand how many tests you’re going to run. It also means that tests can have children. The full implications of that are a subject for another post, but suffice to say here that the reasons to avoid children (how will I know if my test suite is done?) don’t really exist in a Node world, and aren’t really applicable anymore in a browser world.
As a result of giving up something that (as far as I know) every single existing library assumes, we gain a frankly absurd amount of control over how tests are structured and composed. This decision alone vastly simplifies the library while vastly increasing its customizability.
This could be a much longer article, but I want to tie all of this back together into a unifying concept. Whether or not you agree with this principle will likely determine whether or not Distilled is a library you are interested in following, using, or contributing to.
Big ideas should be built out of small ones
Distilled’s entire testing API is 1 method. If you want to build a custom reporter or extend the library, it’s 2 methods. That’s it.
2 methods are all that’s necessary to encapsulate almost the entirety of the modern JS testing ecosystem. Using 2 methods, you can add back beforeEach if it’s something you need. You can add back whatever assertion style you want to use. You can build a crazy runner that scans through five directories if you want to.
By culling these extra features, and by focusing on elegance and extensibility to an almost fanatical degree, Distilled avoids some of the inevitable problems of larger libraries. It doesn’t need to be opinionated because it can be extended in many different directions. It can be flexible and adaptable, yet still keep its total documentation under the length of this very article.
What I’d like to encourage you to do is to take a more drastic, ideological approach to composing large-scale APIs out of small systems that can be combined and extended in interesting and unexpected ways.
If this approach is interesting to you, or you can see value in a testing library that follows it, you can read Distilled’s full documentation here. It is genuinely shorter than this article.
I’ve done some work here hyping the library, but truthfully I’d also love your feedback. There are API decisions I’m still making about Distilled that would be good to run past other developers. And while I’m starting to use it for my own personal projects, Distilled lacks real field testing to identify potential problems.
Got a feature request, concern, or criticism? Raise an issue here.
