What is the Circuit Breaker Pattern?
Think you’re running a web service that requires input and delivers it to another backend service. If backend service is not available for some time, then what kind of fail-proof system you should implement? This is where the Circuit breaker design pattern comes in.
Let’s think about a situation when a request goes to a middleware application and you need to call another remote backend service. If everything goes fine and smooth, the application can forward the request to the backend service and send the response back to the client. But, if the backend service was down, the request could not be completed. In the next few requests from the client also tries to call backend service and getting fail. Here, we can use the Circuit Breaker model to manage backend errors. According to the circuit breaker pattern, the middleware may be in the following possible states.
- Closed State Just like the electronic circuits, the closed state is the operating state. Request a flow through the specified route and no system failures. This status means that the application is up and running. Here, the client’s application was sent back to the backend and the reply was sent back to the client.
- Open State The client request goes to the middleware application, but the backend server is not available. The circuit is in the open state, the same as the electronic circuits. Once the application enters this state, no upcoming request will be sent to the backend server for a specified timeout. Instead of attempting to call the backend service, the application responds to the client with an error message.
- Half-Open State After application in Open State for a given timeout, it comes to the Half-Open state. In this state, the application attempts to send the request back to the backend again and to verify whether the backend service is accessible or not. If it is accessible, the request will move to the closed state. If the backend is still inaccessible, it will go back to the open state.
Why the Circuit Breaker pattern?
Think you have a backend application that executes the Database query and sends the outcome back to the client. Imagine clients are sending 10,000 TPS (transaction per second) to your system. If the backend server crashes and waits for recovery. But the client is still trying to send a huge amount of traffic to the backend server that is still trying to recover. In this type of situation, the system should be designed in such a way that it waits for some time until the service is available. Once backend service is available, send requests.
In the Circuit Breaker pattern, if there is a failure, wait for some timeout and attempt to send a request to the backend service after the timeout. The idea behind this pattern is to encapsulate the logic of managing unexpected mistakes. This pattern is particularly helpful in circumstances such as database migration and software updates.
Circuit Breakers in Microservices
Containers / VMs are dynamic things in the Microservice architecture. The container could vanish, and in the meantime, another container could be launched. The whole concept of Microservices is that there might be a component failure, and the Microservice architecture itself handles all of these failures. Expecting errors will make your software architecture more resilient.
Here are some practical complex scenarios where the NGINX load balancing circuit breakers are used. NGINX Microservices Reference Architecture has a service called “Resizer” that can resize, rotate, shrink an image when the image is uploaded to the system. This resizer operation is a CPU and memory-intensive job that can cause out of memory errors. Here, we can place the Circuit Breaker between the Resizer service and the Image Uploader service.
Here, the Uploader service examines the health status of the Resizer service and checks whether Resizer has enough memory to execute operations. If NGINX has discovered the service to be defective, the uploader will distribute loads among other instants. Once the Crashed Resizer service becomes healthy, NGINX slowly ramps back to the restarted Resizer service.
Implementing the Circuit Breaker pattern
Here is the practical application of the WSO2 Integrator Circuit Breaker Patten. WSO2 Integrator is an open-source middleware platform used to interconnect distinct web services. The integrator is between two endpoints. One endpoint(client) sends a request to the other endpoint(backend) and the integrator performs the required conversion to the request through a sequence of mediators before sending it to the backend endpoint. This scenario can be implemented with the following configurations on the WSO2 Integrator.
<api name="ServiceAPI" context="/cal">
<resource methods="GET">
<inSequence>
<send>
<endpoint key="CircuitBreakerEP"/>
</send>
</inSequence>
<outSequence>
<send/>
</outSequence>
<faultSequence>
<header name="To" action="remove"></header>
<property name="RESPONSE" value="true"></property>
<property name="NO_ENTITY_BODY" scope="axis2" action="remove"></property>
<log level="full"></log>
<payloadFactory media-type="xml">
<format>
<ns:MyResponse xmlns:ns="http://services.samples">
<ns:Error>We can't response you at this time, we will reponse through E-mail soon</ns:Error>
</ns:MyResponse>
</format>
</payloadFactory>
<send/>
</faultSequence>
</resource>
</api>Here we identify the ServiceAPI API that forwards the incoming application to the backend endpoint. Incoming request processed by the mediator `<inSequence>`. Inside this mediator, redirect to the endpoint name “TimeoutEP.” Response to this endpoint will be directed to the `<outSequence >` mediator. Here, the backend reply is sent back to the client. Here, the `< faultSequence >` section is used to record the failure information.
The endpoint definition would be as follows.
<endpoint name="CircuitBreakerEP">
<address uri="http://localhost:9764/CalculatorService-war_1.0.0/services/calculator_service/call">
<suspendOnFailure>
<initialDuration>40000</initialDuration>
</suspendOnFailure>
<markForSuspension>
<errorCodes>101507,101508,101505,101506,101509,101500,101510,101001,101000,101503,101504,101501</errorCodes>
<retriesBeforeSuspension>3</retriesBeforeSuspension>
<retryDelay>400</retryDelay>
</markForSuspension>
<timeout>
<duration>200</duration>
<responseAction>fault</responseAction>
</timeout>
</address>
</endpoint>In the endpoint section, we can define timeout parameters for the Circuit Breaker pattern. Address URI is the backend URI that where request redirects.
- initialDuration: This property is used to define how long to wait until retry backend service in milliseconds.
- retriesBeforeSuspension: Number of retries before going into the suspended state(Open Circuit state).
- retryDelay: delay between two failure calls.
Conclusion
Circuit Breaker design pattern is a basic pattern used in both monolithic and microservice-based deployments. It helps the system prevent sending unnecessary loads to the failed backend service. It provides some delay to backend service in order to recover from errors. It plays a main part in the Microservice architecture as the containers are dynamic and could die at any time. The Middleware platform that is used to manage traffic should be prepared to manage the application when the backend server gets down. Circuit Breaker patterns provide an elegant way to handle errors.
[1] https://www.nginx.com/blog/introducing-the-nginx-microservices-reference-architecture/
[2] https://docs.wso2.com/display/EI6xx/Endpoint+Error+Handling
