Mockito gotcha: Beware of method invocation when stubbing

Here at Kenshoo, we use Mockito quite heavily. It comes in handy for unit-testing services with dependencies, giving us a consistent and exhaustive API for mocking these dependencies with various outputs and behaviors. Usually — this works out fine. But sometimes, the “magic” behind Mockito’s implementation rears its ugly head and creates bugs that are hard to understand.

In this post, I’m going to describe one such issue that we came across recently, where seemingly-correct and standard usage of Mockito threw a NullPointerException for no apparent reason. We’ll explain why it happened, and hopefully learn a thing or two about the inner-workings on Mockito along the way.

Use case: Overriding default stubbing with “thenAnswer”

The offending use case here is quite common: in short, it involves calling thenAnswer twice for the same method. This often happens when we use it once to provide a “default” answer (which applies to most tests in the test class), and then a second time to override that default for a specific test:

Full reproducible example

Imagine you’re testing a service called MyService, which depends on another service called MyOtherService. Here’s a very simple example of such services:

When testing testedMethod, we’d want to mock the behavior of getValue so that we properly isolate our test unit from its dependencies. Here’s a test class doing just that:

Notice that we’re using Mockito’s thenAnswer method, which takes an Answer[T], allowing us to return a value that depends on the actual input arguments the method was called with. In this case, the implementation of answerSizeOfInputPlus:

1. Extracts the first argument of that method invocation and casts it to String.
2. Returns that String’s length plus the value of add.

Inspect the entire class — nothing here looks obviously wrong, right? We expect Mockito to:

1. Create the mock.
2. Stub it to use the result of answerSizeOfInputPlus(2) when called.
3. For one specific test, we stub it again — which “overrides” the stubbed behavior with a new one — answerSizeOfInputPlus(3).

The surprising result is a NullPointerException on the second call to thenAnswer.

What went wrong?

Let’s find the cause for this exception. We’ll focus on these 3 lines:

Line 1

/* 1 */ val dependency: MyOtherService = mock[MyOtherService]

First, we create the mock. That means Mockito creates an anonymous implementation of MyOtherService, where each method is implemented by Mockito as follows:

• Record the fact that the method was called, and with what arguments. By “record”, we mean storing that invocation’s info in some mutable list of invocations.
• If the method has already been “stubbed” (using thenReturn or thenAnswer) with arguments matching the current ones — the method returns that stubbed value.
• Otherwise, the method returns null (or 0 for numeric primitives, false for booleans etc.) or simply does nothing, for a void/Unit method.

Line 2

/* 2 */ when(dependency.getValue(any())).thenAnswer(answerSizeOfInputPlus(2))

We stub the getValue method for the first time. This call shouldn’t actually do anything — it just tells Mockito what to do if and when the method is called. But — look closely! The line contains this expression: dependency.getValue(any()). This is a call to the getValue method! There’s no magic here, the fact that this invocation is part of a when-then expression can’t prevent it from actually being called. But — we haven’t stubbed it yet (that’s what we’re doing now) — so what happens?

Look back at the explanation for line 1: when a method is called, we record that invocation, and if it wasn’t stubbed yet, we return null. So — that’s exactly what happens here: the recording of the method call is critical for the following call to thenAnswer to know what to stub (the last recorded one!), and the returned null value is simply ignored. You can see this in the actual implementation of when:

https://github.com/mockito/mockito/blob/release/1.x/src/org/mockito/internal/MockitoCore.java#L78-L81

The methodCall value, which is the return value from the invocation, is simply not used. In our case, it would be null, and that’s perfectly OK with Mockito.

We move onto the thenAnswer part, which tells Mockito to use the Answer provided by answerSizeOfInputPlus(2) the next time this method gets called (are you already starting to see the wall we’re heading into?…)

Line 3

/* 3 */ when(dependency.getValue(any())).thenAnswer(answerSizeOfInputPlus(3))

We stub the getValue method for the second time. Just like before, stubbing the method requires actually invoking it (again, no magic — dependency.getValue(any()) is a method invocation). But this time, it has already been stubbed! We’ve told Mockito to use the Answer returned by answerSizeOfInputPlus(2), which has the following implementation:

The first thing this does is extract the first argument of that invocation and cast it into a String. But what would that string be? If we look at the invocation, we’ll notice the argument passed is any(), which is one of Mockito’s many “Matchers”. Matchers are used to create stubbing for a wide variety of inputs (as opposed to having to stub the right response for every single possible input). Internally, Matchers operate similarly to Mockito.mock itself: they “record” the desired matching rules (i.e. change some internal state), and their return value is insignificant. In fact, they usually just return null, as we can see for any():

So — the invocation of getValue in line 3 has null as its first argument. That means that inputArg in answerSizeOfInputPlus would be null when it’s executed here. And therefore, the next line (inputArg.length + add) produces a NullPointerException.

To summarize, this error occurs when:

1. We stub a method call using thenAnswer.
2. The Answer we use inspects the invocation’s arguments, assuming they’re non-null.
3. We stub the same method again, inadvertently invoking the method and triggering the first stubbing with a null argument.

Mystery solved, Hurrah! But… what can we do?

Solutions

The easiest solution would be to treat a null argument as a valid option: answerSizeOfInputPlus will check whether inputArg is null (and, for example, simply return 0 if it is) to avoid the error.

But there’s a cleaner solution: Mockito provides an alternative stubbing API via the Mockito.doAnswer static method. If we use that API for the second stubbing (line 3) we’d avoid the exception:

doAnswer(answerSizeOfInputPlus(3)).when(dependency).getValue(any())

Why is this safe?

By calling doAnswer first, we inform Mockito that we’re about to stub the next method call. Hence, Mockito knows that the next invocation isn’t an actual one that should use previous stubbing — and avoids calling any previously-stubbed Answers. This is a clean (yet inexplicable to the naked eye) change that avoids the need to handle nulls in the first place.

In conclusion, as always, open source libraries can be huge productivity boosters and the good ones are often self-explanatory and easy to pick up. But sometimes, there’s no escaping getting to know some of their intricacies, and carefully reading the documentation, Javadocs, or the code itself. Mockito is no exception, as this case shows, but it’s mature enough for the quirks to be rare, and to provide good workarounds for them.