On testing private methods

There is a mantra in software engineering that says you should not write test coverage for private methods (or functions). In this post I am going to explain why I think that this, as many other mantras, is not a universal truth, and sometimes it is totally fine to test private methods.

Best practices are not hard rules

The software industry likes to have mantras like this, the law of this, the principle of that, the code metric you should not violate, the best practice you should adhere to, etc.

I am generally speaking very skeptical when these are presented a priori. Best practices arise, they emerge, they are the result of reflecting about what many people have observed to be characteristics of good software.

Let’s take for example LOCs in a function. When you are presented with a long function, statistically speaking, it is often the case that breaking the function down results in a better understanding of what is going on, you come with good names, the structure becomes declarative instead of needing to be inferred by the reader. The golden rule that follows from that experience is that normally functions are short in a well-written program.

Does that mean you should never write a long function? Fuck, no. Sometimes the natural implementation of a function is just a bit longer than usual. You look at it, you apply your best judgement as a responsible adult, and conclude that it is fine.

With time you develop your own sense of what is a smell and what is not.

Also, time and experience change your nose, oftentimes towards seeking simplicity and best judgement in a case by case basis. The book of rules is back covered by dust, because you have learned their origin, and when they do not apply in your opinion.

Testing private methods

Testing private methods is one of those things. You’ll hear often that you should not test them. Let me break that apart for you.

The purpose of a test suite is to verify the observable behavior of your system.

Why do you write tests? To increase your confidence in that the system is going to function correctly for whoever is going to use it. And that is where the “observable” property is key. How does the system accomplish behaving correctly is irrelevant, the only thing that matters is that from the point of view of its usage, it works.

When you are writing a library, you want the users of your software to have a correct behavior when they use what they can use, which is the public interface of your library. In that sense, statistically speaking, what you want is to cover the public surface of your library. You treat it as a black box. And that is why normally testing private methods or functions is not necessary.

That is, if you are new to testing and you are told to unit test everything, as a rule of thumb, that means testing the public interface rather that writing tests for every single function, public and private.

Why? Again, because the observable behavior is what matters.

But sometimes, exceptionally, testing the public interface is just too complicated. Rare in practice, but when it happens you have to recognize it, test your private implementation, and sleep well.

This week I was precisely in this situation, writing a class that looks like this:

Client code is only concerned with the #load method, but the implementation naturally splits in separate loaders, each of those with their own complexity, edge cases, etc. Too much to be all tested through #load. So do what makes sense for the case at hand: test extensively the private methods, and write some ultra simple safety net thing for the public one.

Some people would extract that to individual classes and use composition. Why? Well, in those classes the methods would be public. My point is that formal visibility per se does not matter, nobody except the umbrella class is going to use the classes, so no matter what the method formal visibility says, you are still testing private interface. Moving to classes is a sidekick that leaves the problem conceptually in the same place.

Also, the impulse to extract to classes in a case like this is that you feel that should be testable. But that is mixing stuff in my view, if you feel the code has to be tested, hell, test it. Define your public interface according to use cases of the API, the public interface and the testable surface often overlap, but they are not generally equal.

Rather, you look at the code. Methods are fine? Excellent, test them.

Design your code according to itself and its public usage. Do not contort your code to please your theoretical test suite desires. The test suite is there to serve the code, it is subject to the code, it has to adapt to the code, not the other way around. The code is the main character in this movie.

Technical constraints

There is a technical point to be made here though. Ruby has a way to bypass visibility via #send and thus you can test private methods from the test suite. Other languages like C or Python just do not have visibility constraints and may depend on conventions as documentation or leading underscores in function names, which allow them to be called from the test suite anyway.

Languages with hard visibility constraints may require other workarounds though. Depending on the constraints composition could be the only way out. In such case the pragmatic solution may be to sacrifice the design of the public interface and declare the auxiliary class to be internal somehow.

Michał Muskała commented on Twitter that in Erlang it is common to write these tests in the modules themselves, so they have access to everything. Clearly a test function does not belong to the public interface of a module, so that is an interesting approach.

Also, Dave Thomas recently published an Elixir library to have private methods be public in the test environment. You could do that also in Ruby easily, but the compromise is that the function becomes public for the entire test suite, which is not going to match the production runtime (not sure personally if that is a good trade-off, in Ruby I never use this technique, but Elixir does not seem to have anything else so it is not that you have many choices).

The C# extension PostSharp (which I knew about via Aitor Guevara) provides a way of declaring access to a whitelist of types, and also project-level visibility, having test coverage of private interfaces as a documented use case.

Parallelism with integration tests

Testing private methods is no different than what is conventionally done with web applications. Web applications are used through their interface, so the ideal would be to write integration tests and period. What happens behind the scenes is irrelevant to verify the system.

In theory you do not care how do you validate a bank account number. What you absolutely want to make sure is that when you enter the account number in a form it passes if valid, and it does not pass otherwise. So, in theory, what you really want to do is to write integration tests.

But in practice, exercising the entire test suite of a bank account validator through a form, with all edge cases etc., and together with perhaps other eight fields that could interact with each other in who knows how many ways, is impractical.

And there it goes your pragmatism, you write an exhaustive I-am-going-to-break-you test suite for the account validator, and then perhaps one positive and one negative examples in integration. You assume that the request ends up calling the validator (if you believed that test suites proved correctness, welcome to the real world, I have yet to see one), and that is your test suite.

And from the observable point of view, unit testing the validator is like testing a private method. It belongs to the private surface. But you sit down, look at the situation, and conclude that is all fine and makes sense from a practical standpoint.

Conclusion

So, folks, if you feel like some of your private functions need to be tested, totally go for it.