Promises, Chaos, Async and Tests
A look a why you need a little randomness in time for your asynchronous tests.
I’m sold on the importance of randomness in tests. Why unit test on ‘hello world’ when Math.random().toString(36).substring(2, 15) is so much more interesting. Not only does it make your results more exciting to glance through, but it’s also provides some protection against result-bias; it stops you from designing functions that just happen to work with ‘hello world’.
This lesson was rammed-home on me today, when I discovered that our unit tests for an asynchronous feature were in fact tied to a specific pattern of asynchronous’ity.
My job for today was to replace our current locking library with a new one; one more capable on a distributed system. In theory it should have been a find, replace, refactor job. However, when it came to running the tests everything started failing. The reason? The new library was more asynchronous that the last- or, more specifically, the time to return was less predictable.
It turns out that the tests written against the old system were working only because the old system was ‘barely’ asynchronous; the majority of requests would be fulfilled in the same tick of node’s event loop. This was further exacerbated by some mocked libraries behaving synchronously lower down the stack.
This led to two failures- firstly, the tests didn’t actually verify what they had set out to do, and, more importantly; had shaped a solution that didn’t actually meet the requirements. Test Driven Development, it turns out, is as susceptible to garbage-in-garbage-out as anything else!
So today I learned a trick for making your tests chaotic in asynchronous-time. Wrap your mocking functions with the same degree of randomness you should treat your data with.
For example:
It’s too easy to test against ideal conditions, and so adding a bit of chaos is always a great way to help catch those edge cases and really test your programming assumptions. And that, as I learnt today, goes for time too.
