10 tips I wish I had known before I started testing

I never heard a word about testing at college, not even in my early work. Several years after, I started “to play” with tests by my own but it was not until 2011 that I began to write tests in a professional way. Since then I have asked myself so many times how could I work for 10 years without automated testing.

That’s why the first piece of advice I would have liked to receive is: Start testing now, please! Why? Keep reading…

1. Start testing now, please!

Your test suite will help you to have better code. Not the next week or in a year time, it will help you immediately because in order to write a test you have to review your code once again and this time from a different perspective. Sometimes you will be forced to improve your code (e.g. add an interface) so it can be tested and this is something good.

Your code will become more robust. How many times have you avoided to touch a piece of code just because you don’t want to take the risk of breaking something else? Imagine instead; you make the changes and then you trigger a build to check that your code is stable. If the risk of breaking your code is very low you will refactor more often, so your code will be more up to date and healthier.

You will save time. How could it be possible if you are going to spend time writing tests? Because you will have less bugs and you will detect them earlier so it will be easier and faster to fix your code. Less emergency patches in production, less customers complains, less stress. Also, because one thing leads to another and you will need to integrate your tests as part of your build and then you will need a Continuous Integration environment where you can build your project every day and so on. Your tests are just the first step, the more automated your work is, the fewer mistakes you will make.

It’s up to you. Even if your business culture doesn’t believe in it, or you don’t have time assigned to it or you work in a start-up, you should test!

2. Create meaningful tests

Every single test should bring some value. Ask yourself: What does this test cover? What is its purpose? The answer must be clear not only for you, but for everyone in the team. If not, consider refactoring it, or even deleting it.

Each test must have a single goal. The goal could be something small like checking a method that adds up 2 numbers or something bigger like checking the registration flow. When the target is big, like in the second case, you could focus on ensuring that the registration steps work well together within a single test, executing one step after another and just checking at the very end that the entire flow succeed. Of course, in most of the cases this is not enough, but you can create more tests to cover more deeply every step. In short, don’t try to cover too many things with one test because it will be more difficult to understand and maintain. You could always split a complex logic and cover it with smaller tests.

It’s also very important for tests to be easy to be read. Readability means you can understand the goal, the scope and what is being checked so you can focus on what is important in each case. It is not always easy, for instance when you are covering a functionality that has many paths, you will need at least one test case per path and in some cases they could be very similar so it’s nice to understand the difference between every case. You could have a meaningful test method name as a starting point (e.g. registerUserWhenUserUnderAgeThenFail) or add some comments but most of the time the best practice is to have a clean code.

But please, never ever create a test just to pretend you are testing a piece of code. This will generate a false sense of security.

3. Create independent tests

Let’s say you are testing the “register user” flow, so as part of your testing strategy, you have to check if the new user is created in the database. In order to do that, you could call the “find user by id” method that you already have in the code or write a new independent method which checks the same thing.

A friend of mine prefers the first option. The advantage of this approach is that you are checking in the same test the “register user” flow and also the “find user by id” method. In other words you are covering more code with your test. But this is also a disadvantage, because if your “find user by id” method has a bug this test will be broken even when the logic behind the “register user” is correct.

I personally prefer the second approach and yes, I spend more time writing a new independent method to check if user is in the database. Why? Mainly to avoid a false positive when you have to fix a broken test.

It is also important to remove any dependency between your tests, so that the execution order does not interfere with the success. Another good reason is that you could run your tests in parallel.

4. Think globally and be smart

Over time your test suite will grow and grow. This is essentially good but it can bring some collateral problems like your building time will grow as well. So you need to think globally and design a testing strategy and find the right balance: unit test vs. integration tests, mocking vs. real code, partial tests vs. end-to-end tests.

You could for instance, maximise the number of low-layer tests because they are faster or because you don’t need to rollout your application in order to execute them. This will allow you to cover as much as possible as soon as possible and create just the happy cases for the end-to-end tests. In any case, don’t take it as an axiom; your testing strategy will depend on your business requirements and your software complexity.

Testing by layer is another good approach. You code one layer, then you test it and only then you move to the next layer knowing the previous one is working fine and so on. Let’s say you have following stack: UI, controllers, services and persistence. For this case, I would start coding the persistence methods (as they don’t depend on any other layer) with the corresponding tests and only after I would move to coding the service layer with their tests. If you find an error at this point, it should be in the service layer as the persistence layer is safe. This way you should save time and headaches. It’s important to highlight that you need to know in advance the whole sequence so you don’t have to refactor everything just because you realised in the top layers that you need something that you don’t have. In some cases, it could be better to write the whole skeleton to be sure all the layers can work well together before you start testing anything.

Avoid duplicated tests: if you already covered something, there is no point to test it again, unless you consider this part very important and you want to be really sure that it works correctly. Then test it again but do it differently.

Reuse your test code as much as possible. Don’t copy the same code across your tests. Instead, try to create code that can be reused so when the logic changes, you only need to update one place.

You have to be smart in order to cover as much as you can with your tests (not only the happy cases), spending as less time as possible taking also into account your maintaining time and your building time.

5. Unit tests vs Integration tests

Unit tests are for testing a part of the code (e.g. a class) in isolation. You have to mock any kind of collaboration with the rest of the code. In this way, you could guarantee this part behaves as intended.

Advantages: These tests are very fast, they are small and you can find problems early.

Imagine you have to test a method like this, with 2 public inputs and 1 public output:

int divide(int a, int b)

This is a perfect case. It would be enough to create tests based on the method’s signature, assigning different values to “a” and “b” and checking the expected response. Your tests will not directly depend on the implementation, so there will be less chance they will be broken next time you refactor your code. The method is like a black box for your tests.

Disadvantages: It is not always easy to mock and sometimes it could be very tricky to create a proper test.

Imagine instead you have to test a method with no public output:

void process(Item item)

In this case you could tend to rely too much on the code structure as there is no obvious way to check an expected response. This is not good because the test is more likely to break with future changes.

On the other hand, we have the integration tests. As their name implies, they are for testing several parts of the code working together. You could mock or stub some dependency (e.g. with a third party library) but most of the code is the real one.

Advantages: You could ensure all parties involved work well together. You could focus on the inputs and outputs even when they are not public and avoid internal dependencies with the code structure.

Disadvantages: These tests are not as fast as the unit tests and the scope is bigger. They could be more complex.

Both approaches are valid and the use of them will depend on your software design and your development life cycle.

Refactoring is something very common in my daily day as I work using Scrum methodology. In this iterative and incremental context, I prefer to implement unit tests only for the parts of the code that rarely change (e.g. mature code or framework code) or for very sensitive code that needs to be tested in isolation (e.g. testing an aspect). For the rest, I prefer integration tests mainly because I can guarantee a functionality works and I don’t have to update the test every time I refactor the code.

6. Generate your seed data

Generate your seed data means you have a piece of code that generates the data you will need to run a test. You could have a single data set for all the tests but I prefer to share the same set between few tests so every time something changes, it only affects a small number of tests. In addition, as these sets will be smaller, they will be easier to maintain.

Why should you generate your seed data instead of writing it manually? Well, this is not straight-forward, I will try to explain it based on my mistakes.

I started writing a single seed data set for all my integration tests manually. Very soon I realised that I had to update the seed data almost every time I had to implement new test as I didn’t have any data for it or because the data didn’t match my new requirements. Then when I updated the seed data for the new tests I used to break the old ones. Terrible approach!

The good part, I realised this very soon. Then, I started to write the seed data, manually of course, but this time a different set for each different flow. And, when I had to implement a new test, no problem at all, I wrote a new seed data for the new requirements. Happy days!

But time goes by and very often a new requirement breaks many of the existing seed data because the business logic has changed. When you have few tests is OK, but when your test suite has grown then you have to spend more and more (and more…) time only updating the existing seed data and guess what? Yes, manually!

So far, three main errors:

· The seed data is not in a central place, so you have to go test by test to fix the broken seed data.

· They were written manually, so they don’t follow the same pattern and are harder to correct.

· I realised too late. I already had lot of tests using different manual seed data and changing the strategy at that time had a higher cost (time, money and a lot of code to migrate).

What could you do instead?

· Generate your seed data from a central place (to minimize the impact of your future changes).

· Encapsulate the complexity and interdependency of your seed data inside the generator (the less code you have outside the generator, the less you will have to maintain).

· Easy to use (try to have a friendly human public interface).

For instance, the next code generates the SQL for 1 staff and 10 UK-based users from Java:

SeedDataGenerator sdg = new SeedDataGenerator();
sdg.getStaffMother().build();
sdg.getUserMother().withCountry(“UK”).build(10);
sdg.generateSql();

I know that at the beginning it may look easier just to do it manually but if you work in a project for a while, you will face this problem at some point.

If you don’t agree with me, or if you do but you don’t have time, don’t try to create the next seed-data-generator framework that will change the world. You could start instead with a very simple piece of code that encapsulate in a single place the seed data you need so far and improve it as you go on.

7. Create a rich seed data

Ideally your seed data should be as rich and as varied as your production data but in most of the cases this is very difficult to achieve. You can maximize the entropy of your seed data in many ways but following patterns at the same time so you have a rich data but also a testable one. You could for example:

· Assign a different base number to each primary key. Instead of starting with 1, 2, 3 you could assign a big base number to each entity so they can’t overlap. For instance: 1000 to your staff (1001, 1002, 1003…) and 2000 to your users (2001, 2002, 2003) so if by mistake you set the wrong value you will realize immediately.

· Generate different data but following a pattern. Random data is more difficult to test. For instance you could have unique names by combining the entity name as a prefix with the order number. For staff names: staffOne, staffTwo; for user names: userOne, userTwo.

· Avoid using the current time for your date time values, instead you could use fixed but incremental date time values. For instance, if you have to record the date of issue, you could use “2017/10/08 17:06:01” for the first row and then increment a second for every new row, so you can predict the value and also order by date and always get the same result.

8. Review all your tests from time to time

I know you don’t have time, me neither. I won’t ask you to take some days off and review all your old tests. No way! But what about each time you are fixing a broken test, or you are adding a new test into an existent file, you take 5 minutes and review the whole file.

Am I asking too much? Well, I know sometimes it will take you more than 5 minutes to bring those old tests up to date, but in the worst case you could create a new ticket in your backlog.

9. Love your tests as much as you love your code

You should apply the same quality standards you already have for your base code to your test code mostly because you will have to maintain it. Keep in mind you will have to update your tests every time your base code changes, so write them in a way that will be easy to understand.

Don’t consider your test code as secondary. You should take care of it in the same way you do with your code, also it should be part of the code reviews.

Don’t leave the tests for your QA team. Don’t leave the tests for later. Every time you write new code, or change an existing one, you must write or update the corresponding tests. Your base code is incomplete without tests. Everybody needs to understand that, not just developers and you should help me to persuade the whole World.

If you’ve read this far it’s because you love coding and you want to improve yourself. Then why not loving testing too? Call me naïve if you wish but I cannot conceive coding without testing any more. Both things are like the faces of the same coin.

10. The last reason

This article is open. Would you like to add one?