Write automated tests at three levels

Matt Stephens
Oct 19, 2019 · 6 min read
Image credit: Alyssa Scott

There are of course almost as many types of tests as there are classifications of bugs. However I find it useful to focus on three types in particular while developing a new feature:

  • Unit Tests
  • Component Tests
  • Acceptance Tests

Given the diverse, fast-paced world of IT, these three test categories have gained many synonyms, each of which often brings with it a slightly different emphasis; and often the lines blur between each category.

But by way of a quick introduction:

  • Unit tests are written by developers as they’re coding. These tests tend to be quite fine-grained — one test covers a single function or class, basically a unit of functionality or (to put another way) a single responsibility.
  • Component tests are also written by developers as they’re coding. These tests are broader-grained than unit tests — each test covers more code, typically a group of closely related classes.
  • Acceptance tests are specified by BAs, or by software testers who are in direct contact with the BAs. These are end-to-end tests, covering slices of functionality that might run from one end of the system (e.g. a REST endpoint) to the other (perhaps a database query) and back.

So, three types of test that each operate at a different level, or scope.

But why test at these three levels in particular? And, you’d be right to ask, isn’t there too much crossover between them, i.e. duplicated effort?

The answer is that they each provide different benefits that complement each other rather well. There tends to be far less duplication than you might think, as they’re testing for different things.

Photo by Brigitta Schneiter on Unsplash

Think of a unit test as a fishing rod — precise, aimed at a particular area of water, designed to catch a single fish. An acceptance test by comparison is a trawler net cast wide. It’ll catch hundreds of fish, including some possibly unexpected “prizes”, though many smaller fish will pass through the net without even being noticed. A component test is the “in-between” size, probably like the hand-held net cast into the waves by a Maui fisherman. It’s targeted but will still catch a fair number of fish, and the mesh is small enough to catch the tiddlers that otherwise would have wriggled through.

But enough already with this strained metaphor… let’s dive deeper to see how each level of testing provides its own set of benefits.

Why write unit tests?

Here are some good reasons to write your tests at the “unit” level:

  • To prevent bugs within the single unit under test
  • To show that the function/method you’re writing works as intended, according to its interface (i.e. contract-based testing)
  • To better understand the design, and to drive the design to an extent
  • To question the code

So, pretty essential then! However you’ll notice that all three of these benefits occur right at the point where you’re writing the code that the unit test is aimed at. Unit tests do provide feedback into the design—with the coding you’re now right at the coalface, discovering what’s possible and what isn’t. However, some unit tests’ “window of utility” soon passes, and they can quickly turn from an asset to a liability.

It’s tempting to keep unit tests around as a blanket form of regression test. However (thinking back to the fishing rod analogy), this really isn’t their area of strength: in fact they kind of suck at regression testing. They don’t cast a wide enough net. All they really test is that a function has been implemented in a particular way. They’re more effective as a temporary, in situ development tool than as regression tests.

Why write component tests?

Here are some good reasons to write your tests at the “component” level:

  • To prevent bugs being introduced within the component under test
  • To show that the function/method you’re writing works as intended, according to the requirements (ideally, happy-path and unhappy-path scenarios broken down from the user stories)
  • To better understand the design, and to drive the design to an extent
  • To validate the design
  • To verify that business logic (e.g. a complex state machine that runs across several classes) works as specified
  • To provide a blanket of tests, a regression safety net, that can be re-run regularly in order to catch new bugs as early as possible

There’s some crossover with unit tests, which isn’t surprising as they’re both written by developers while they’re programming. But there are also plenty of benefits that you just don’t really get with finer-grained unit tests — or at least, not nearly as effectively.

By the way, as I mentioned earlier, a component test’s job is to validate a cluster of functionality — a closely related group of functions, classes or responsibilities that together fulfill one particular item of system behaviour. But how to identify the scope of an individual component? If you’re a fan of Domain Driven Design (DDD), the component under test often matches a single aggregate, or a bounded context if it’s contained in a single microservice. Worth thinking about!

Overall, I’ve found component tests to be way more effective than unit tests — easier to write (more “black box” than “white box”); and you don’t have to write as many, as each one is broader in scope and so covers more code. They cast a wider net than unit tests.

So that just leaves testing at the broadest level of the three…

Why write acceptance tests?

Here are some good reasons to write acceptance tests:

  • To question the depth or shallowness of the requirements
  • To verify that business logic works as specified
  • To ensure error conditions are handled, e.g. a suitable error message is displayed, or a transaction rolls back correctly so data integrity is maintained
  • To provide a blanket of tests, a regression safety net, that can be re-run regularly

There’s some overlap with component tests, but this is actually where component tests can be really useful —if, just like acceptance tests, you drive them from the business requirements/story scenarios rather than the code.

But back to acceptance tests…

To be truly effective, acceptance test scenarios should be specified by the subject matter experts involved in the project — the BAs, or at least by testers who are in direct contact with the BAs. But it’s often the developers who will need to turn the acceptance test scripts into something that runs (e.g. implement the Java or Ruby code behind a BDD Gherkin scenario).

Acceptance tests, being end-to-end, cast the widest net of all; so you would correctly expect them to be the most effective form of regression test. The downside is that they usually take longer to run than component tests, so you either write fewer of them, or run them less often.

Which one is just right?

As you can see, while testing at three different levels you might be testing the same codebase beneath it all, but you’re achieving different goals at each level.

A nice way to think of these differences is that unit tests verify that you’re writing the code right, while acceptance tests verify that the code is doing the right thing. Component tests occupy a crucial middle-ground that is often missed. In fact, while testing at all three levels is important, I tend to focus far more attention on testing at the component level than the other two… and you should too!

Interested to know more? Check out my upcoming book, Domain Oriented Testing (due out sometime in early 2020).

By the way, if you’re a software developer and you haven’t yet taken this unit testing survey, please do so — it should take less than a minute, and you’ll help form a more accurate picture of unit testing practices across our industry… thanks!

Matt Stephens

Written by

DomainOrientedTesting.com — Articles on: Domain Driven Design, unit testing, TDD, DOT, software architecture, clean coding (and code cleaning)

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade