Tests? Why and What

walter Torricos
Sep 1, 2018 · 8 min read

Why should I care

Well, do you find difficult to perform a change in your application? Do you fear you will break something else after you perform a change? Do you find hard to upgrade the version of a library? Do you consider risky to refactor your code? Does the QA team find bugs frequently? Easy, YOU DON’T HAVE TESTS!!

I agree with Michael Feathers when he defines legacy applications:

“Legacy Applications is any application that isn’t covered by tests” Michael Feathers — Working effectively with legacy applications

As you may infer from the questions at the beginning legacy applications are applications were we have FEAR of making changes and the cause of this is always the lack of tests.

Legacy == FEAR

The objective of having tests is to lose the fear and to make our lives easier in the long run. No matter the application you are working on, at some point you will have to perform changes and there are two ways to make them:

  1. Edit and Pray. You can be really careful on every change, carefully reviewing all the parts that can be affected, identifying all the paths that can go wrong and ultimately performing your change. The thing is that things can and will go wrong with this approach, the last part you thought it could break will break.
  2. Cover and Modify. On the other hand you can have your code covered by tests and this way when you perform a change you will know that something broke immediately.

Your tests suite is a test harness. As the name suggest is similar to a climbing harness that will prevent you from falling to the ground and hurting yourself. But not only that, your test harness should really give you confidence that your system works. In other words:

“A test harness should be as nasty and vicious as real world systems will be. The test harness should leave scars on the system under test (SUT). Its job is to make the SUT cynical” Released it — Michael Nygard

To be in the same page making a SUT cynical means that we want our applications to expect bad things to happen and never be surprised when they do. Like one of the go proverbs says

“Don’t just check errors, handle them gracefully”. go proverb

I see, but what is a test?

The first definition I’m going to provide you is a little vague but will be complementing it through the rest of the article:

“A test is something that checks that a piece of code works as expected when certain conditions are met”

Now let’s see some characteristics of good tests:

  • Fully automated. All your test suite runs automatically with a trigger. Most IDEs today allow you to run your test suites with a click or a command, and of course continuous integration tools allow you to run your tests when you push some change to your repo.
  • Deterministic. Your tests must always provide the same results. Tests that pass most of the times but fail under certain conditions (like running in a different environment) are not fully deterministic thus are not good tests and should be fixed.
  • Unordered. They can run in any other meaning they are completely independent of one another.
  • Maintainable. Tests are as important as the rest of your code base, so they must be written as such. They must be readable, well structured, etc. The only principle you can allow yourself to break is the DRY principle (Don’t Repeat Yourself). As all the tests must be independent is hard to avoid breaking the DRY principle without adding some accidental complexity.
  • Trustworthy. You can trust your tests. After running them you feel very confident that you didn’t introduced any bug. Having a high coverage will also help you with this.
  • Not fragile!!!!!!. Test that are fragile are test that easily break after you make some change (Maybe you know to much about the implementation). To avoid having fragile tests always test everything as a black box when possible. Please read this characteristic one, two and more times if needed. If your tests break after small changes they are fragile. I’ve seen a lot of this and people writing them without realizing it.

Testing Categories

Tests also have categories, in order to review them I’m gonna introduce you to the test pyramid.

Testing Pyramid

The test pyramid can look a bit complex at the beginning so we are going to break it into smaller pieces and then explain all of them.

The first thing to note is that we want more of what’s in the bottom and less of what’s in the top. The question is why? Notice that the pyramid is divided in two parts, one of them is labeled “QA/Automation Tests” and the other one “Dev Tests”

  • Dev Tests. are tests that must be implemented by developers and that are able to run in a dev machine.
  • QA/Automation Tests. They are usually implemented by the QA/Automation team, this tests are heavier and they may need additional infrastructure to run.

Now let’s focus on the green arrow on the right. The turtle and the rabbit represent the cost in time, the closer you are to the top the slower your tests will run. The dollar sign and the moneybox represent the cost of implementation, the closer to the top the more expensive the implementation.

Wait a minute what does it mean that the tests on the top are expensive to implement? Well they are more expensive because they depend on many moving parts, like different services, databases, etc. Additionally small changes like an update in the look and feel of a page may break an E2E test as the selectors will no longer be able to select a button thus a test will fail giving a false negative. As you can see the tests at the top of the pyramid tend to be more fragile by nature.

Then why in hell would I want to write the tests on the top? Well the tests on the top give you more confidence that your system works.

Now that we understand the pyramid is time to review every category in detail.

Exploratory

I won’t go into too much details for this one, basically is manual testing that is done in order to find new cases to automate.

E2E

E2E (End to End testing) Is automation testing through the UI. The entire system is manipulated as a black box through GUIs and APIs. They test the gaps between services, and they also ensure additional infrastructure such as firewalls, proxies or load balancers are correctly configured.

They are hard to maintain and write due to its many moving parts, so here are some best practices for implementing them:

  • Apply a time budget to keep them small.
  • Model around the most important use cases.
  • Make the test data independent.

QA component Tests

In a microservice architecture, the components are the services themselves. Tests are driven from a consumer perspective. This tests interact at an API level and use real Databases and external services, like it can be seen represented in the image below.

QA Component Testing

This tests can be really helpful specially for regression testing, so it’s ok if you have a lot of them but always be sure to apply a time budget as in the E2E tests.

Integration Tests

I had a lot of discussions about what integration tests are, so before explaining them let’s see some definitions:

“Two or more independent software modules as a group.” The art of unit testing — Roy Osheerove

“They are slow tests because they talk to a database, communicate over the network, touch the file system or you need to set up and environment to run it.” Working effectively with legacy code — Michael Feathers

“An integration test verifies the communication paths and interactions between components to detect interface defects. They are usually used to verify interactions with external components such as other microservices, data stores and caches.” Testing Microservices — Martin Fowler

I’m not going against or in favor of any of those definitions, but the one we are going to use for this article is the second one, as it’s the one that fits best with our testing pyramid.

So basically what we want to test here are all of our proxies to the outside world. For example our complex queries to the database, or the interaction with an SFTP server, etc.

Integration Tests

As you can imagine this tests will run slow, this is why they are at the top of the Dev Tests, they will be really useful to test that our queries to the database are returning the correct resources, and that we can correctly communicate to other services, even that we can parse correctly a file. But this things need to be checked only once and then we can mock them.

Dev Component Tests

I really like this tests as they are the fastest ones that also give us a lot of security that our application is working as expected. They test the interactions between the different modules of our application as they represent complete use cases. The only drawback with them is that they may not be that cheap to implement, first of all your application needs to start in testing mode, yes you heard right, so you need to design for testing too!!

Basically they will test complete use cases similar to the QA Component Tests but the difference is that these tests will use test doubles or in memory implementations for all the external resources, this way we are going to keep them fast while covering complete use cases.

Dev Component Tests

Note on Dev Component Tests: They are no silver bullets and according to your use cases there are occasions where test doubles or in memory implementations won’t be enough, for those cases it’s ok to use some real dependencies, like a real DB but try to have as few as possible.

Unit Tests

They are the smallest piece of software that can be tested and we can divide them in two:

  • Sociable unit testing. Test the behavior of modules (they may include many classes that live in the same module), they treat the SUT as a black box.
  • Solitary unit testing. Test the unit under test completely in isolation by replacing dependencies with test doubles.

My recommendation is to have as many sociable tests as you can. Try to use test doubles only to mock external dependencies. Of course there may be scenarios where mocking all your dependencies may be useful especially if you have complex logical paths that you want to test. In any case this is the hardest part as you have to use your best judgement to choose which one fits better.

Conclusion

I hope this article was useful to you and that you can understand better what are unit tests, how they classify and their importance. In the future I’ll be posting examples, best practices and patterns that will help you test your code.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade