Circuit Breaker Pattern —Resilience4j

In distributed systems, it’s common to have remote calls. Systems such as microservices architecture based software consists of tens and hundreds of standalone services which have relations and collaborations with each other. This interaction is possible through inter-process communication.

In microservices architecture, while using inter-process communications such as REST — a synchronous communication style — you have to consider the availability of the services. Remote calls may fail due to unavailability of another dependent service and it’s the most important difference between remote calls and in-memory calls.

Sometimes the process of responding to the incoming request takes a long time and cause timeout error. It will be getting worse if you have multiple callers on an unresponsive service, so the system can run out of resource.

The solution to all of these problems is circuit breaker pattern. This pattern is used to detect failures in a system and prevent the failure from constantly recurring. Now let’s see how does it work.

How does circuit breaker works?

A circuit breaker consists of three states :

Closed — This state indicates that everything is working fine.

Open — This state indicates that a failure has happened. By changing the state from closed to open, it returns a specific response directly without processing the business logic.

With the occurrence of failure, the state of circuit breaker will turn into open state and avoids processing the logic. So there must be a mechanism to rollback state from open to closed state when everything goes back to normal. There is now the third state present here.

Half-open — This state indicates that the circuit breaker is ready to recall the unresponsive service as trial. If the problem with the service is fixed, it will turn the state from open to closed.

Notice that it’s not required to open the circuit breaker for every failure. It is not useful while occurring regular business errors. You can return any value to the client when the state of circuit breaker is open, it can be an exception or a cached data.

Resilience4j

As the official documentation describes, Resilience4j is a lightweight, easy-to-use fault tolerance library inspired by Netflix Hystrix, but designed for Java 8 and functional programming. Let’s implement a sample project with this powerful library.

This library have many features such as circuit breaker, bulkhead, rate-limiter, retry, time-limiter, cache and etc. Now let’s implement the circuit breaker pattern in microservices architecture using Resilience4j, Spring boot and Docker.

Implementation

First, go to spring initializr and generate two projects. One for the client and second for the server.

The server is responsible to expose a RESTful API which takes a string value, then returns the length of that value along side the value itself. The server needs the following dependencies; spring web and lombok.

The client also takes a string value, then performs a remote call to the server and receives the response from underlying service (server) and return it to its users. The client needs the following dependencies; spring web, lombok and resilience4j-circuit-breaker.

Let’s get into the implementation of server application first. As i said, the server exposes an API which takes a parameter and returns the length of that parameter. So we can have to entities; request and response.

Request

Response

After defining the models, let’s define the controller layer. In this layer, we have to create a post API which takes a request entity and an integer which is a value in milliseconds, then we have to pass both of them to the service layer.

For the service layer, we have to create an interface and define an abstract behavior of our business logic which is counting the letters of a string value.

Then we have to create a class for implementing the logic of our server application.

The Thread.sleep(wait) has been implemented to test the circuit breaker pattern later in the client application.

Now let’s get into the implementation of the client application. The client application is responsible to make a remote call to the server and return the received response to its users. Both entities — request and response — can be used here too. Except those two entities, we have another one called MainModel. This entity is a wrapper for sending response to the users.

As well as the controller layer in server application, the controller layer in the client also has a post API which takes a request entity and an integer which is a value in milliseconds, then calls the logic in the service layer. In the process() method of our controller, we wrap the response in MainModel entity.

In the abstraction of service layer, we have to specify two methods, one for calling the underlying service (server), another for processing the remote call with the help of circuit breaker.

In the implementation of service layer, we have to implement the methods above. But before that, let’s define the circuit breaker configurations. We can define it as a bean to inject the CircuitBreaker later.

The CircuitBreakerConfig class is used to define our custom circuit breaker configuration.

• slidingWindowType() is used to record the outcome of calls when the circuit breaker is in closed state.
• minimumNumberOfCalls() is used to configure the minimum number of calls which are required before the circuit breaker can calculate the error rate or slow call rate.
• slidingWindowType() is used to record the outcome of calls when the circuit breaker is in closed state.
• slowCallDurationThreshold() specifies the duration that a call is considered as slow.
• slowCallRateThreshold() circuit breaker considers a call as slow when the call duration is greater than slowCallDurationThreshold(). When the percentage of slow calls is equal or greater the threshold, the circuit breaker state will turn into open state.
• waitDurationInOpenState() is the time that the circuit breaker should wait before transitioning from open to half-open.
• permittedNumberOfCallsInHalfOpenState() configures the number of permitted calls when the circuit breaker is in half-open state.

Time based circuit breaker

The time-based circuit breaker turns into open state, if 80% of requests out of 10 requests fail.

Count based circuit breaker

The count-based circuit breaker turns into open state, if 80% of requests out of 10 requests takes more than 2 second to respond.

Now let’s go back into the implementation of service layer. We have to implement two methods here. The callServer() methods makes a remote call to the server with RestTemplate. In the process() method, we define a supplier which calls the callServer() method, then return the response to the controller layer.

Now let’s dockerize both microservice applications — the server and the client.

Server’s Dockerfile.

Client’s Dockerfile.

The definition of docker-compose.yaml

Everything is ready and the last thing we have to do is testing our applications. Let’s create server and client containers using docker-compose. We can do this with the help of the commands below.

Notice that the configuration of our circuit breaker is count-based. If 80% of requests out of 10 requests takes more than 2 second to respond, the circuit breaker will turn into open state. We can test this scenario by calling the client’s API 10 time with delay of 3000ms.

After calling the service 10 times with the delay of 3000ms, the console logs show that the state of circuit breaker turned into open.

Also the response of the API in open state, will always be as follows. It directly returns the specified response (in this case an exception) without processing the logic which is calling the server API.

After 10 seconds state will turn from open state into half-open state. By calling the API 5 more times with the delay (the wait parameter) of zero, everything will go back to normal and the state will turn into closed.

Conclusion

The availability of services in distributed systems such as microservices architecture based systems is so important. The circuit breaker pattern will solve the problems related to availability in your system and it helps you serve a better experience for the users of your system.

Git repository

The project is available in this git repository.