In this article, we will learn the fundamentals of one of the must pattern in the Microservices world — Circuit Breaker. We will do the sample implementation based on Spring Boot, Spring Cloud & Resilience4j. This is the sixth part of our Spring Boot Microservices series.

Lal Verma

Follow

Sep 21, 2020 · 10 min read

What is Circuit Breaker?

As the name suggests, the pattern derives its inspiration from the electrical switches, which are designed to protect an electrical circuit from damage, caused by excess current from an overload.

When a particular microservice or resource is not responding, this pattern helps in registering the fault, switching off the communication, and restoring it back when the service is ready to serve the requests. This helps the microservice ecosystem in multiple ways —

1. It handles the service failure and exits gracefully
2. It helps in reducing the overload on the service, which is already stressed
3. It stops the spread of failure across other services

Let's try to understand the pattern with a real world example. We are working in the e-commerce domain and our system is built on Microservices Architecture. For simplicity, let's consider two services.

The first is Product Catalog Service, responsible to manage product lifecycle through — create, update, delete, and get operations. And the second is Product Inventory Service, responsible to manage product inventory through add, update, and get operations.

Here’s the typical call from our e-commerce store. It calls the Product Catalog Service to get the product details. It has the basic product information including title, description, price but not the available quantity. To get the quantity, it calls the Product Inventory Service.

What if the Product Inventory Service is down? If you don’t handle this scenario, it will result in the failure of Product Catalog Service too. The situation can get worse if the Product Inventory Service is slow in responding. This means it's already consuming resources to its limit.

Let's say, another service called Order Management Service is consuming the Product Inventory Service at this point in time. If the Inventory Service is bombarded with multiple calls from Product Catalog Service, at the same time, it will result in the failure of Order Management Service too.

This is an example of cascading failure, which can propagate to the whole system if not handled correctly. Simple exception handling is not good enough in this case. The Circuit Breaker pattern provides an elegant, maintainable, and flexible approach to handle such failures.

How does the pattern work?

Resilience4j has done a good job in explaining how the pattern works. The Circuit Breaker pattern is implemented with three states: CLOSED, OPEN and HALF-OPEN.

The Circuit Breaker sits right in the middle of the call to external service/resource. In our case when Product Catalog Service calls the Product Inventory Service, the call will go through the Circuit Breaker component.

Circuit Breaker will be in a CLOSED state by default. Let's say the configured threshold is 10%. This means if 10 out of 100 requests are failed, it will be assumed that the failure threshold is reached. At this point, the circuit breaker will move to the OPEN state. After a while, it will move to the HALF-OPEN state to check the status of the Product Inventory Service. At this point, it will open the communication channel at a limited rate. If the rate of failure continues to be above the threshold value (10%), it will move to the OPEN state again. If not, it will move to the CLOSED state and the expected communication will be resumed.

With this pattern in place, you can always exit gracefully and control the rate of transaction flows as per the service availability.

Sample Implementation

Let's do the sample implementation to see how it works on the ground. We will be doing the implementation based on Spring Boot, Spring Cloud, and Resilience4j. This implementation is also based on Project Reactor, which means we will be using Spring Web flux and Spring Cloud Reactive Circuit Breaker. I have divided the exercise into three following parts.

1. Building Product Inventory Service — We will implement add, update, and get inventory APIs of this service.
2. Building Product Catalog Service — We will implement create, update, delete, and get product APIs of this service. Our primary focus will be on GET API though.
3. Implementing Circuit Breaker — We will implement the circuit breaker in the get Product Details API of Product Catalog Service.

Also, please ensure that you have Java 11 and Maven 3.x. Any version of Java beyond 1.8 should work but I did not validate for other versions.

Building Product Inventory Service

As discussed our Product Inventory Service will consist of addProductInventory, updateProductInventory, and getProductInventory APIs. We will use MongoDB for data persistence. Let's go to the spring initializer and generate the project structure.

Add Spring Reactive Web, Spring Data Reactive MongoDB as dependencies. Generate, download, and unpack the archive at your local system.

Let's implement the restful APIs in ProductInventoryService.java . Update the file with the following code.

@RestController
public class ProductInventoryService {    @Autowired
    private ReactiveMongoTemplate mongoTemplate;    @PostMapping("/inventory")
    public Mono < ProductInventory > addProductInventory(@RequestBody ProductInventory product) {
        return mongoTemplate.insert(product);
    }    @PutMapping("/inventory")
    public Mono < ProductInventory > updateProductInventory(@RequestBody ProductInventory product) {
        return mongoTemplate.save(product);
    }    @GetMapping("/inventory/{id}")
    public Mono < ProductInventory > getProductInventory(@PathVariable String id) {
        return mongoTemplate.findById(id, ProductInventory.class);
    }}

We are using ProductInventory as the data object, so let's create another class for this — ProductInventory.java

public class ProductInventory {
    @Id
    private String productId;
    private int quantity;   //getters and setters ...  }

We will be running this service on port 8082. Update src/main/resources/application.properties file with the server port and MongoDB connection details. I am using the hosted MongoDB instance, available as part of the free trial from MongoDB. You can choose any other MongoDB instance, as per your choice.

server.port=8082
spring.data.mongodb.uri=mongodb+srv://xxx-user:xxx-pwdY@cluster0.nrsv6.gcp.mongodb.net/ecommerce

Our Product Inventory Service is ready to function now. Start the service with the maven command mvn spring-boot:run . You can use addProductInventory API to insert some sample records. Here is the sample post request I used.

{
 "productId": "test-product-123",
 "quantity": 30
}

Building Product Catalog Service

Similar to the Product Inventory Service, create the Product Catalog Service. We are going to add two more dependencies — Spring Cloud Circuit Breaker (with Resilience4j), and Spring Boot Actuator (optional).

Let's create the class ProductCatalogService.java to implement our restful APIs as below.

@RestController
public class ProductCatalogService {    @Autowired
    private ReactiveMongoTemplate mongoTemplate;    @Autowired
    private WebClient webClient;    @Autowired
    private ReactiveCircuitBreakerFactory cbFactory;    @GetMapping("/product/{id}")
    public Mono < ProductDetails > getProductDetails(@PathVariable String id) {Mono < ProductDetails > productDetailsMono = mongoTemplate.findById(id, ProductDetails.class);Mono < ProductInventory > inventoryMono = webClient.get().uri("http://localhost:8082/inventory/" + id).retrieve()
            .bodyToMono(ProductInventory.class);Mono < ProductDetails > mergedProductDetails = Mono.zip(productDetailsMono, inventoryMono,
            (productDetails, inventory) - > {
                productDetails.setInventory(inventory);
                return productDetails;
            });return mergedProductDetails;
    }  @PostMapping("/product")
  public Mono<Product> addProduct(@RequestBody Product product){
   return mongoTemplate.insert(product);
  }
  ...}

If you take a close look at the getProductDetails API, we are first getting the basic product details and then calling the Product Inventory Service (running at port 8082) to get the inventory details. We are combining both the results before returning the response.

Update the src/main/resources/application.properties with the MongoDB connection details as discussed above. Start the service with mvn spring-boot:run command. The Product Catalog Service is ready for use now.

Insert the sample product records with the createProduct API. Here is the sample post request, I used.

{
 "id":"test-product-123",
 "title":"test-product-1",
 "desc":"test product 1",
 "imagePath":"gc://image-path",
 "unitPrice":10.00
}

Let's call the getProductDetails API by accessinghttp://localhost:8080/cb/product/test-product-123 . We should get the result something like —

{
 "id":"test-product-123",
 "title":"test product for circuit breaker",
 "desc":"test product updated",
 "imagePath":"gc://image-path",
 "unitPrice":10.0,
 "inventory":{
   "productId":"test-product-123",
   "quantity":30
 }
}

Great! So our base platform is set. Both the services are working perfectly as expected. It's time to implement the Circuit Breaker pattern.

Implementing Circuit Breaker

Before implementing this pattern, let's turn down the Product Inventory Service and see how it impacts. While this service is not running, access the getProductDetails API of Product Catalog Service, browsing the page at http://localhost:8081/cb/product/test-product-123 . You will see an error, somewhat similar to this —

[70118947-6] There was an unexpected error (type=Internal Server Error, status=500).

This will be the result if we do not handle the service failures. With the Circuit Breaker pattern, we will be implementing a fallback mechanism. In our case, it's going to be “returning a blank object”. With this, the store can still display the product details, with no availability and we will be handling the worst-case scenario gracefully.

Let's update the getProductDetails API as below.

@RestController
public class ProductCatalogService {
   
    @Autowired
    private ReactiveCircuitBreakerFactory cbFactory;    @GetMapping("/product/{id}")
    public Mono < ProductDetails > getProductDetailsV2(@PathVariable String id) {
        Mono < ProductDetails > productDetailsMono = mongoTemplate.findById(id, ProductDetails.class);        Mono < ProductInventory > inventoryMono = webClient.get()
            .uri("http://localhost:8082/inventory/" + id)
            .retrieve()
            .bodyToMono(ProductInventory.class)
            .transform(
                it - > cbFactory.create("inventory-service")
                .run(it,
                    throwable - > {
                        return Mono.just(new ProductInventory());
                    }
                )
            );Mono < ProductDetails > mergedProductDetails = Mono.zip(productDetailsMono, inventoryMono, (productDetails, inventory) - > {
            productDetails.setInventory(inventory);
            return productDetails;
        });return mergedProductDetails;
    }}

In the highlighted code, we are updating the call for Product Inventory Service with the following behavior change —

When the exception occurs in the call, its returning blank ProductInventory object.

With the help of ReactiveCircuitBreakerFactory , we are creating the Circuit Breaker instance based on inventory-service configuration. We need to include the following configuration in src/main/resources/application.yml—

resilience4j.circuitbreaker:
  instances:
    inventory-service:
      failureRateThreshold: 50
      minimumNumberOfCalls: 20
      slidingWindowType: TIME_BASED
      slidingWindowSize: 10
      waitDurationInOpenState: 50s
      permittedNumberOfCallsInHalfOpenState: 3

We will touch upon all these attributes in a while. Before that, let's restart our Product Catalog Service and access the getProductDetails API. Voila! you got the product details even though the product inventory service is down.

{
  "id": "test-product-123",
  "title": "test-product-123_1",
  "desc": "test product updated",
  "imagePath": "gc://image-path",
  "unitPrice": 10,
  "inventory": {
    "productId": null,
    "quantity": 0
  }
}

Though Spring Cloud and Resilience4j made it look easy, a lot of magic is happening behind the scenes, to make the pattern work.

As discussed earlier, the Circuit Breaker will be in a CLOSED state initially. As we hit the getProductDetails API first time and Product Inventory Service is not available, it executes the fallback operation — to return the blank object. Let's see how each of the attributes are defined in application.yml are helping us.

• failureRateThreshold —The circuit breaker will continue to be in a CLOSED state till the failure rate threshold reaches. In our case, this value is 50%, which means if 5 out of 10 requests are failed, the threshold will be reached. This will move the circuit breaker in an OPEN state, which means it will stop the unnecessary network calls to call Product Inventory Service.

• minimumNumberOfCalls — This attribute ensures the failure rate is calculated once a minimum number of calls are executed. In our case, 20 requests must be catered for before the failure rate calculation starts.
• slidingWindowType — This attribute configures the failure rate calculation mechanism. The failure rate can be either calculated based on time or the count of requests. For instance, we can say consider the request in the last 5 minutes or consider the last 50 requests. In our case, we are defining the time-based window — TIME_BASED
• slidingWindowSize — With this attribute, we are defining the window size, which is 10 seconds in our case. If the 10 requests processed in the 10-second window and 4 of them are failed, the failure rate will be 40%.
• permittedNumberOfCallsInHalfOpenState — Once the circuit breaker moves to OPEN state, after a while it moves to HALF-OPEN state. In this state, it communicates with the product inventory service in a limited manner. This attribute defines this limit. In our case, the limit is 3, which means only 3 requests will be processed in a 10-second window.

Hurray! We just implemented a real-life example of the Circuit Breaker pattern. What next …?

Next Steps

The primary objective of this article is to develop a high-level understanding of how the Circuit Breaker pattern works. The article covered the basic implementation of this pattern but as you adopt it in your services, you might need to customize it from multiple aspects.

We just used one of the modules of the Spring Cloud Circuit Breaker library. It supports other implementations too including Netflix Hystrix, Sentinel, and Spring Retry. Check out the Spring Cloud Circuit Breaker documentation for more details.

Our implementation was based on reactive code, but Spring Cloud supports non-reactive implementations (for e.g Spring MVC) as well. You can again find more details on this at Spring Cloud Circuit Breaker page.

Resilience4j provides a configurable and cleaner option to implement a Circuit Breaker for java based applications. We used only a few of the attributes for configuration. You can refer to the complete list of attributes, available with Resilience documentation.

Also, Circuit Breaker is just one feature offered by the library. It also offers RateLimiter, Retry, Bulkhead, Time Limiter, and Cache decorators. You can combine multiple decorators to make the service calls. For instance Bulkhead, RateLimiter, and Retry decorator can be combined with the CircuitBreaker decorator. For more details visit the documentation here.

You can browse1. Next Exercise - Kubernetes-ization.
 
2. Complete Series - Spring Boot Microservices - Learning through Examples3. Previous Exercise - Implementing Api GatewayComplete source code of the sample implementation is available at Github