Let's say that we have a function "add index to database" that accepts an
index which is a primitive. The purpose of the function is to run a complex operation before calling another function inside it called "database persist", passing that same
Let's say, for the sake of the argument, that:
- This is a piece of a legacy code with low test coverage. We need to test as much as possible to reduce the chances of breaking how that function currently behaves.
- For the reason above, we can't do Test-First.
- We want to test that the "database persist" function will be called with the index input we provide.
Given those constraints, we need to mock the "database persist" function and check if that function will be properly called with the value we want:
Everything looks fine, except for the fact we introduced a potential false positive in the test code.
What if inside the body of the function there's a change that reassigns the
index argument to a value of 5 that is coincidentally the same value we are using for the "fake index" input in the test?
In this case, the test will pass because it's not really checking if the "database persist" function is being called with the
index input we provided, but rather that it's being called with
5, assuming an input of
In the future, if somebody overrides the
index variable (which is assumed to be an invariant throughout the execution of the "add index to database" function), then there will be a non-deterministic test that will keep passing even if the mocked function is not really being called with the input.
That is not a very useful test.
This problem happens because when we test the arguments of the mocked "database persist" function we are comparing the argument value, instead of comparing the reference.
Testing that the inner mocked function is called with the primitive input of the parent function can lead to false positives because we are comparing the value, not the reference
This seems obvious in hindsight, but it can easily pass unseen, even in code review.
One solution for this problem is, instead of passing a primitive as the "fake input" in the test, we pass the instance of some dummy object so that when we compare with the "fake input" we can be sure that we are comparing with the actual reference, instead of comparing with the value.
For this solution, it doesn’t matter which type the "add index to database" function accepts. We can pass an Object Literal just for the sake of holding the reference in the variable so that we can have a proper deterministic test:
Passing a dummy Object Literal will be more robust and future-proof. But only if we can duck type the interface of the original “index” argument to an Object Literal. The "add index to database" function might be using the
index in a way it forces a caller to call it with a number argument, like when it uses the input for arithmetic operations.
If you don’t understand the meaning of “interface” in this context, take a look at this post.
If there are arithmetic operations like
index + index, those cannot be duck typed to an Object Literal and therefore we will need to use a new instance that provides the same interface of a number, like
That will work, because now we are creating a specific instance, and checking against that instance, instead of against the primitive value.
That will also allow the code to treat the input as a primitive for most use cases throughout the test, so that if the arithmetic operations change in the future, then the test will still pass legitimately.
The instance approach works perfectly when we can substitute the primitive, but it doesn't work when we are passing values such as
undefined, which have no equivalent way to be passed by reference.
Leveraging duck typing to pass by reference instead of passing by value doesn't work when we pass something that doesn't have a reference equivalent that can be duck typed
The example below shows a false positive being introduced when we test the “add index to database” function with an
In that case, a solution for the false positive can be something like property-based generative testing. It will test for the mocking call property using many samples instead of relying on a single one. Another solution can be triangulation.
Unfortunately, that means adding more abstractions into your tests and relying on statistics, but considering the weight of the benefits and the low likelihood of your code reassigning the same types that are being generated, it might not be all that bad.
Sometimes mocking in unit tests can be tricky. Understanding exactly what we want to test and how the language comparison system works is a must to avoid some dangerous and subtle traps.
Generative testing and triangulation can be an alternative solution that use statistics to improve the determinism of a mock call, but as most of the things in software development, there are no silver bullets.
Maybe what you can do is not to use mock at all.
This article became a talk: Mocking And False Positives. The slides have other examples and more details.