My Tests are Broken Again, Part 1: Understanding the Problem
The importance of decoupling our tests from implementation details is getting more attention recently; the popularity of libraries that advocate good practices is growing larger than those that do not. And yet, I would like to argue that most of us — even if we are “testing our projects as our users would use it” — most of us are still writing tests that are coupled to implementation details.
Let’s find out why!
The Components Architecture
It is a common practice nowadays to develop applications using a Components Architecture. Whenever we separate our code into different repositories, we basically create another component in our architecture. Each component can then be deployed independently. Each component — when it makes sense to do so — can be owned by a different team.
An example components architecture might be:
Component Tests
Sometimes known as integration tests, component tests verify that our application does what we expect it to do. But unlike unit tests, they do it “from the outside.” Component tests are not aware of the internals of our application.
Every two (or more) components that are communicating with each other have established a contract between them. The contract between two components is the structure and schema of the APIs they’re using to communicate with one another. In many cases, the contract is implemented using REST, gRPC or GraphQL. But it might as well be a Kafka message — one component is the producer, whereas the other is the consumer.
Component tests ought to be coupled to the contracts established between the component-under-test and its neighboring components.
Let’s take the front-end component of our application as an example.
The neighboring component of the front-end is its back-end. While testing the front-end, we should couple our tests only to the way the data arrives from its back-end.
We should not mock a Redux selector and then assert that our application renders the correct React components. Should the implementation of some components change — for example, if a component stopped using this selector — our test will certainly break!
Instead, we should test our application “from outside”: first we intercept the API call that fetches the initial data for the application (I use msw for this). Then we will render the entire application — thereby ignoring all the implementation details — and, finally, we will make assertions solely on the things that users can (or should) see: buttons, text, etc.
Given state X, users should see button A but not button B. Such tests will not be aware (and, therefore, would be decoupled from) the internals of our application.
But why is that good, anyway?
The Benefits of Component Testing
I find component testing to be a very effective approach. These tests have two benefits over unit tests and e2e tests:
- Unlike unit tests, they are decoupled from the implementation.
- Unlike e2e tests, they allow a short test-code-refactor cycle.
Component tests provide the above benefits while still creating a fairly strong feeling of confidence. When our component tests are all green, we usually feel that it is safe to deploy.
Component tests are also relatively easy to maintain, as, again, they are decoupled from the implementation.
So, if the above is true, then we should make the shift from the Testing Pyramid toward the Testing Diamond:
We would still write unit tests and e2e tests from time to time, but our focus will be on testing the components “from outside”—each component as an independent unit.
What makes a good component test?
A good indication of whether you’ve written good component tests is that when refactoring your code, your tests do not need to be adapted to these changes.
The Deterministic Application
So, are we writing good component tests?
Before answering this question, I’d like to explain another (hopefully the last!) definition:
Every application is a pure, deterministic function.
Let’s take a to-dos application as an example. Our initial state is a pending to-do. The action is clicking on the to-do. Clicking on a to-do should mark it as done. No matter how many times we render the application with the state “has one pending to-do,” and then click on that to-do, the result would remain the same: the item would be marked as “done.”
In other words: given a state, when performing an action, the result is deterministic.
Let’s see why this is so important.
Re-Inspecting Our Testing Diamond
I would like to argue that we are not truly testing our applications “from outside.” Our tests are not completely decoupled from the implementation.
Why? Well, as we said earlier, our application is a pure, deterministic function. Our application-function is defined by three elements: Initial state, Action, Derived State. When testing front-end applications “from outside,” we usually focus on interacting with the application as users would. We then make assertions based on what users should see (or experience) as a result of the action under testing. But we have completely ignored the Initial State.
An example should help us understand why.
I recently worked on supporting video meetings (via Zoom) within Fiverr’s communication platform: the Inbox. One of the requirements was that when video meetings end, the duration of the meeting must be calculated and shown to users.
It looks like this:
We can see that the duration of the meeting above was 30 minutes.
One way of testing this behavior would be:
Naturally, this is a very simplified way to write the test.
If we wanted to refactor something in the internal implementation of this video meeting card — for example, using MobX instead of Redux — our test would not need to change. Great!
But notice this — our test is still coupled to the implementation. We are currently storing the duration in minutes as a number. But should we, for some reason, decide to store the start- and end-time of each meeting and then calculate the duration in the browser, our tests will break.
They would need to be re-written like this:
So, going back to the idea that our application is a pure function—a function that receives an initial state and an action and outputs a derived state—it seems that while we are testing our code, we are performing actions as our users would perform them; we are making assertions upon the derived state as our users would do so; but we do not create the initial state as our users would!
I apologize if I have shattered the illusion that your tests were decoupled from the implementation details.
Recap
We’ve discussed the benefits of Component Tests: they are (well, almost) decoupled from the implementation, they provide a greater sense of security, and they enable a speedy test-code-refactor cycle.
But the way we write our tests today is not perfect. We realized that even though we are “testing our projects as our users would use it,” our tests are still coupled to the implementation: they are strongly coupled to the way we chose to model the contracts between the component-under-test and its neighboring components.
In part 2, we’ll explore a new technique that will decouple our tests from these contracts, making our Testing Diamond doubly shiny!
Fiverr is hiring in Tel Aviv and Kyiv. Learn more about us here.
