I have been working with the reactor library for quite a while now and this blog is about a dilemma that every developer might face while writing tests for non-blocking reactive code. This blog will help you to set some rules in your team or squad on whether to use block or StepVerifier based on specific cases. Hence, this will clear some doubts about which one to use where and when.
Before reading this blog you must be familiar with Mono & Flux in the reactor library and also have some prior experience of writing tests using StepVerifier or block, in order to understand the comparison between both and to relate your experiences with it.
Let's first try to build an understanding of what block or StepVerfier is and what it provides in terms of testing a reactive code.
A block is an API that is defined on both Mono and Flux, It internally calls subscribe and immediately blocks the execution of flow until a signal(output) is received. It is not recommended to use a block in the main production code as it blocks the main thread, due to which it can cause a cascading effect where all further calls to the method are blocked and one will have to wait for the execution of the previous subscription on that method. Blocking the main thread in the reactor by doing a subscribe or block takes away all the benefits of writing non-blocking reactive code. Still, if someone wants to write a block in a reactive world, they can use the subscribeOn() API on Mono/Flux which takes Schedulers(eg. elastic) that makes the execution of Mono/Flux on different scheduler threads and execute it asynchronously without blocking the main thread.
But except for the production source code, developers can choose to write tests by using a block() API as it is not going to affect the performance of the application in any way.
Using block() API to write tests for a Mono and Flux:
Above is a small example of how you can use a block to write a test for reactive publishers. In this example, a Mono of Person[0|1] and Flux of Person[n] are blocked for the output signal, and then the output is tested.
To test the exceptions one can use a block() API on Mono/Flux that executes until the error signal is received and throws the exception. Therefore, in the case of Mono that is either empty or one, the exception can be caught and asserted with the output of the method, but in case of Flux which is a continuous stream of data performing a block() operation, will lead to subscription of all the elements which are emitted by the Flux that will result in an error. As we can see in the example given below, only the error that is thrown is tested and we further won’t be able to test the data emitted before the exception.
We have seen examples of how one can use the block method to wait for the signal of publisher Mono/Flux and write tests on the basis of the output. But, there are some limitations for using block() to test reactive code and they are the following:
- How to test Flux which is emitting a stream of data. [As we have seen above if there is an exception in between the data stream, block() returns the exception and we are not able to test data before exception]
- Delays can not be tested using block [ Mono.delay(Duration.ofSeconds(12))]
- Testing the Publishers which are subscribed to different schedulers using subscribeOn() API can not be tested using a block.
In a nutshell, a block() API can only test the output of the publisher by subscribing to it, but it cannot test the reactive aspect of the code written using the reactor library. To test the reactive aspect of the code, the reactor provides a test library that has a declarative way of writing reactor tests using StepVerifier.
StepVerifier is provided by reactor test dependency that supports the testing of the reactive aspects of the code. It provides a declarative way to expect events that will occur on the subscription of an async publisher and it only gets triggered after terminal methods have been invoked i.e verify(). With the StepVerifier, we can define our expectations of published elements in terms of what elements we expect and what happens when our stream completes. StepVerifier subscribes to the publisher and consumes data as a stream from the emitting Mono/FLux.
Following is an example of using StepVerifier to write tests for a Mono and Flux:
StepVerifier provides rich API to test delays and subscriptions on different schedulers using withVirtualTime() API, which was not possible to test in case of a block(). StepVerifier also supports writing tests for the publishers that emit errors. In the case of Flux, using a block() API for the testing of data is not possible where the data is emitted before a publisher has thrown an exception. Unlike block() API, in StepVerifier we can expect the publisher to emit data until the exception is thrown, as we have known that the publisher is closed after an exception is thrown.
Following is an example of StepVerifier to test Exception:
As we have seen that StepVerifier is not only testing the output of the reactive code but it is also testing the reactive aspect of the code. It tests how the async publisher emits data and behaves when it is subscribed by a subscriber. With all the ease that StepVerifier provides for testing reactor code it also has some shortcomings with the syntax, for writing tests. For the assertion of a simple output Mono[0|1], StepVerifier makes assertion noisy and quite difficult to read. One has to even remember to write a verify() in the chain at the end, else the test will pass without execution of the code.
As you can see that there are pros and cons of both block() and StepVerifier testing patterns. Hence, it is necessary to define a pattern or set of rules which can guide us on how to use StepVerifier and block().
In order to decide which patterns to use, we can try to answer the following questions which will provide a clear expectation from the tests we are going to write:
- Are we trying to test the reactive aspect of the code or just the output of the code?
- In which of the patterns we find clarity based on the 3 A’s of testing i.e Arrange, Act, and Assert, in order to make the test understandable?
- What are the limitations of the block() API over StepVerifier in testing reactive code?
- Which API is more fluent for writing tests in case of Exception?
If you try answering all these questions above, you will find the answers to “what” and “where”. So, just give it a thought before reading the following answers:
- block() tests the output of the code and not the reactive aspect. In such a case where we are concerned about testing the output of the code, rather than the reactive aspect of the code we can use a block() instead of StepVerifier as it is easy to write and the tests are more readable.
- The assertion library for a block() pattern is better organized in terms of 3 A’s pattern i.e Arrange, Act, and Assert than StepVerifier. In StepVerfier while testing a method call for a mock class or even while testing a Mono output one has to write expectation in the form of chained methods, unlike assert which in my opinion decreases the readability of the tests. Also, if you forget to write the terminal step i.e verify() in case of StepVerifier, the code will not get executed and the test will go green. So, the developer has to be very careful about calling verify at end of the chain.
- There are some aspects of reactive code that can not be tested by using block() API. In such cases, one should use StepVerifier when we are testing a Flux of data or subscription delays or subscriptions on different Schedulers, etc, where the developer is bound to use StepVerifier.
- To verify exception by using block() API you need to use assertThatThrownBy API in assertions library that catches the exception. With the use of an assertion API, error message and instance of the exception can be asserted. StepVerifier also provides assertions on exception by expectError() API and supports the assertion of the element before errors are thrown in a Flux of elements that can not be achieved by block(). So, for the assertion of exception, StepVerifier is better than a block() as it can assert both Mono/Flux.
I have formed all these opinions through gaining experience by writing tests for reactor code. And this should not be assumed as a standard for writing tests for the reactor. Choosing an appropriate pattern totally depends upon the team's comfort with the assertions library. If the team is not comfortable enough with assertions they can choose to stick with StepVerfier there is no harm in it. This blog guides you to gain knowledge of when to use what depending on the situation.
Thank’s for reading :)