System under test

Go placidly amid the noise and haste,
and remember what peace there may be in silence

Recently, I published this tweet:

Properties

  • Isolated — tests should return the same results regardless of the order in which they are run.
  • Composable — if tests are isolated, then I can run 1 or 10 or 100 or 1,000,000 and get the same results.
  • Fast — tests should run quickly.
  • Inspiring — passing the tests should inspire confidence
  • Writable — tests should be cheap to write relative to the cost of the code being tested.
  • Readable — tests should be comprehensible for reader, invoking the motivation for writing this particular test.
  • Behavioral — tests should be sensitive to changes in the behavior of the code under test. If the behavior changes, the test result should change.
  • Structure-insensitive — tests should not change their result if the structure of the code changes.
  • Automated — tests should run without human intervention.
  • Specific — if a test fails, the cause of the failure should be obvious.
  • Deterministic — if nothing changes, the test result shouldn’t change.
  • Predictive — if the tests all pass, then the code under test should be suitable for production.

Combinations

Here are some attractors in the space of all possible tests:

  • Acceptance. Make tests readable for non-programmers even if you give up some speed and specificity.
  • Monitoring. Yes, monitoring is a test too, or rather tests and monitoring are both forms of feedback. Monitoring abandons being predictive and to some degree being automated (modulo alerting).

So What?

Look at the last test you wrote. Which properties does it have? Which does it lack? Is that the tradeoff you want to make?

Kent is a long-time programmer who also sings, plays guitar, plays poker, and makes cheese. He works at Gusto, the small business people platform.

Kent is a long-time programmer who also sings, plays guitar, plays poker, and makes cheese. He works at Gusto, the small business people platform.