Test it Now or Regret It Later…

In this blog post we will talk about testing. I know, Real Engineers can prove that their code is correct from First Principles so why even talk about it? Well because: it is important, underappreciated and absolutely priceless, if done well. Now, will it guarantee that your startup sells for millions or billions of dollars? No. It alone cannot guarantee that. But it can reduce the likelihood that your users go away because their experience with your software is interrupted by crashes, inconsistent behavior or other byzantine issues.

Now, I know that most people don’t want to read a blog post about testing. I thought that a funny cartoon would be appropriate but my drawing skillz are so awful that no one was ever able to identity that it was a bowl of fruit in my sketch. I briefly considered interpretive dance, but that too seemed insufficient to convey the necessary information.

So after much consideration, I settled on memes as a way to discuss the objections to, techniques for and benefits of testing.

Objection #1 — Who Needs Tests Anyway?

You need tests to verify the key behaviors of your system. The mapping of your mental solution to a programming language is always fraught with translation errors and subtle misunderstandings. Testing can help put a check on that.

Objection #2 — Not Enough Time to Write Tests

A little bit more time now, will save you a lot of time later. You can short-circuit a lot of live debugging in Production environments and bug triage if you write some tests up front. If you want to bake a cake with the extra time, that is up to you.

Objection #3 — Testing is Hard

If testing seems hard, you are probably doing something wrong or you don’t understand the problem well enough.

Objection #4 — Works at My Desk

You may think that it works at your desk today… but will it tomorrow, next week or next year? There is only one way to guarantee that — lock it down with a test.

Objection #5 — Code is Fine — Tests Keep Breaking

So, the tests seem to randomly fail? That can be a bummer… it can also be a sign that perhaps there is an aspect to the system that you misunderstand (poorly written tests aside). This is always worth pursuing.

Objection #6 — Test in Production

I’ve heard this one alot. Just test in production. Sounds lazy and lazy can be good, but lazy always runs out of luck — sometimes in dramatic fashion. Production can be used for validation of behaviors already qualified in a non-production environment.

You can run experiments in production to see what resonates with users, but that it very different from just pushing some code to production that hasn’t been through any kind of testing already.

Ready for a Change?

If you are ready to give testing a chance, the next few sections should help get you on a good path.

Technique #1 — Design for Testability

Crazy but true. To really make something testable, you need to consider how it will be tested during the design phase. What access points do I need to trigger the behaviors that I need to verify? Much easier to address this up front.

Technique #2 — Continuous Integration

If some aspect of the system is important, then you should write tests for it. If you want to get the most benefits from those tests, you must run them on some regular basis. Hence you need a continuous integration system. There are lots out there, just pick one and go.

Technique #3 — Test the Critical Paths — Not All Paths

Once people get the testing bug, you find them wanting to test everything — which can be a waste of time and give testing a bad name. You want to test the things that are critical to your system. The things that would cause you a lot of grief if they didn’t work well.

Technique #4 — Add Tests as Needed

You don’t have to add all of the tests on day one. As you find defects and fix them, you can add tests to ensure that if the bug ever comes back you will know about it.

Benefit #1 — Cover Your Backside

When you have tests that cover the key aspects of your system, you will rest easy knowing that even though bugs always exist they are likely going to be in the fringes or less important parts of you system.

Benefit #2 — Conserve Energy for what matters

Since you know that the important behaviors are covered with tests, you have one less thing to worry about. You can take that extra peace of mind and focus on being more proactive in general.

Benefit #3 — Tests Yield Leverage

This one is important. Tests are leverage because they codify the important behaviors. This means that if you have decent tests you can be very aggressive about how you evolve implementation(s) of the code that makes up your system because you have a standard reference for how it should behave.

Conclusion

There are lots of objections that engineers put up to testing, most of which can be easily addressed. Ultimately, you can create a culture around testing by designing systems for testability, adding continuous integration, testing the critical paths and then adding tests to cover newly found bugs. With these techniques in place, you will gain peace of mind, have extra energy to be more proactive and gain additional leverage on your system.

It was because of the leverage afforded by test coverage that we felt confident in our migration from AWS to GCP — which we will start discussing in our next post.