Is unit testing the biggest duplication of effort in human history?

Here’s a thought exercise. You are a JavaScript, Ruby, Python, or PHP developer. Consequently, your language has no compile-time enforcement of, well, almost anything. In fact, your language doesn’t even require a compile step. In some cases, you won’t know about syntax errors until you actually run the code

You are a responsible developer, so you write tests. Because that’s what responsible developers do.

Continuing this thought exercise: What do your tests, well, test? Probably cases like these, among others:

• Ensure functions behave properly when given invalid arguments, like null or unexpected types.
• Ensure optional function parameters default to the right values when not specified by callers.
• Ensure the function exists in the right module and is callable.

Of course, you will test a lot of other cases too, like whether the code does what it’s supposed to do. In my experience, those tests are valuable and necessary, regardless of language. But that’s not what I’m talking about here.

As a bonus, your tests also provide these implicit test cases:

• Ensure the code has no syntax errors.
• Ensure the code is runnable by all the runtimes you care about (e.g., different browsers).
• Ensure the code is using underlying libraries correctly (e.g., complicated ORM APIs).

A linter might give you some of this benefit, but in my experience, it will never quite be enough.

Developers often strive for 100% test coverage, which means they use tools to keep track of which lines of code get executed while their tests are running. 100% means that all your code gets executed by at least one of your tests.

With languages like this, I usually shoot for 100% test coverage. Anything less, and this little proverbial devil appears on my shoulder and says:

You don’t have a compiler. Your tests are your compiler. You must get 100% coverage, or you are shipping un-compiled code, and you are bad.

I tend to listen to this devil, because I do not want to be a Bad Developer.

“Hey big guy, your code needs 100% test coverage, or you are bad”

Once in a while, I take some deep, cleansing breaths and seek enlightenment. When I do this, I realize that I am complicit in quite possibly the biggest duplication of effort in human history.

You see, I used to write a lot of C++ and Java code. These languages have this thing called a “compiler”. This is a tool that checks your code for syntax errors before you can run it. It’s really quite novel in today’s world.

You can think of a compiler as a simple unit test system that gives you some guarantees:

• 100% of your code is guaranteed to be syntax error free.
• 100% of function calls are guaranteed to call functions that actually exist.
• If an underlying library’s public API changes, your compiler will guarantee you know about it before your code runs.
• If you rename a function, your compiler will tell you if any of that function’s call sites haven’t been updated to the new name.

And yet, for every function I write in JavaSript/Python/Ruby/PHP, I have to write unit tests to get these guarantees. No, it’s worse than that. For every function call site, I have to write another test that executes the call site.

For example, if I have a function that is called from 10 places in my code, I must test not only the function, but all 10 of those call sites as well.

That’s O(N) folks.

Oh, and if a future developer refactors my code, and my tests no longer exercise the same code paths, I will silently lose the benefit of those tests. Sorry bro.

The brogrammer says sorry with a sympathetic shrug.

Does a compiler completely replace unit tests? No, that’s not my point.

My point is that when we choose to use a language with no compiler, and we compensate by writing lots of tests, we are duplicating the effort of a few very smart compiler authors. Think about it. If I am writing Go code, there is a small mountain of tests that I will never have to write. The Go compiler authors effectively already wrote them for me, and for every Go project that will ever exist.

The Go gopher says “you’re welcome”.

Funny story from the Go trenches: A few years ago, a friend of mine switched from JavaScript to Go. After writing some code, he tried to write a test to assert that it would behave properly when passed an invalid type. He fought against the compiler until he finally realized this wasn’t even possible.

I’m not saying Go is the One True Language. This concept applies to any language with a compiler. Even languages with an arguably terrible type system, like C++, still give you every benefit I’ve discussed. You don’t have to be a smug Haskell developer to benefit from this concept.

Every time you write a test, think about whether you are implementing a constraint that a compiler would give you for free.

Then take a moment to contemplate your life choices. Think about all those smart and helpful compiler authors. They’re trying to help. Maybe we should let them.

Epilogue

Seriously, you don’t need to read any more of this article.

I warned you.

Lucky for JavaScript developers, the community saw this problem coming and staved it off with tools like Flow and TypeScript. But because these tools are pretty much gradually typed, you can never be certain that every line of code in your project is actually getting all the benefits of the type system. Yes, they are helpful, but because they are interoperable with (or a superset of) JavaScript, you don’t have the 100% safety net that a traditional compiled language would give you.

More advanced type systems can take this concept even further. For example, Ada can provide compile-time enforcement of the value of integers. It can even prevent indexing into an array with an integer that is known to be too large for that array. At compile time.

Haskell has some really powerful type enforcement concepts as well. The `Maybe` type, for example, tells the compiler that a value may contain nothing (similar to null), and any code that uses that value must be aware of this, or it will produce a compiler error.

JavaScript build tools like webpack and Browserify can help a lot with this problem, even if you don’t have any compile time type enforcement. For example, if you have a syntax error in your JavaScript, webpack will spew an error and not output your bundle. That’s pretty handy.

Congratulations to those who read between the lines to see this article for what it is: a thinly veiled rant against dynamically typed languages. I really don’t hate these languages, but the testing tradeoff can be pretty costly. Of course, there are other tradeoffs which make statically typed languages costly: build times, verbose interfaces, build paths, etc. Some statically typed languages (Go) deal with these problems better than others (C++).