Creating Circuit Breaker with 100 Lines of Code

Edward Huang
Feb 13 · 7 min read

“If you want to avoid your entire service going down because one component is down, you should use a circuit breaker,” Kevin told me when I try to design one of our microservices. We talked in front of the ramen shop on a cold winter night in New York City in 2019.

I asked him, “What is it exactly, circuit breaker?”, “Dude! It is a popular microservice pattern. Search it up!”

I pull out my phone and start typing “Circuit Breaker.” In brief, it a mechanism that helps improve the resiliency of your services. I thought to myself, I need to understand more about the tool of this. Therefore, I determined to go back that day, do more research, and implement my own circuit breaker version.

Surprise, surprise — as always, I procrastinated. Fast forward 2 years later, I have learned so much about distributed systems, Scala, and functional programming — and I would like to share the what and the how of circuit breaker mechanism. In this article, I would like to share how I created a circuit breaker library using scala and leveraging the concurrent primitive types.

What is Circuit Breaker

Although the retriable error is prevalent to create resilience in a distributed system, too much retry may actually hurt your service. If you dedicate a thread that keeps retrying the external service, and the service B that is calling your service keeps retrying their calls, it may exhaust all the resources in your services and causes it to go down. This situation usually can escalate to multiple services that depend on the service that is down.

Provided By Author

Therefore, one way to mitigate this is to have some sort of proxy that will be smart about telling your service to stop the retry — because we know that another service is down, there is no need to keep retrying.

This is where the circuit breaker is useful. You can make it as a wrapper on a function or a full service — it has an internal state machine to determine if you need to retry the other service or not. You usually create a proxy for that service to advise if the service needs to send the request or not based on the number of failures. If the current state in the circuit breaker says — “Don’t Retry,” then the service can simply return an error or default message and not occupy a thread for retry.

High-Level Implementation of Circuit Breaker

You start at the Close State. Each time when the call fails, it will increment the error count.

Once that error count hits a certain threshold, the state machine transforms to an Open State. In the Open State, the circuit breaker will advise the service not to do any calls because it is assumed that the other service is down. Therefore, each incoming call to the service will simply reject it.

The breaker waits a couple of times because it transitioned to the Half-Open state. The Half-Open state is where the breaker will tell the service to try on the call again — just to see if the market returns any 5xx. If the call is successful, then it will transition to a Closed State. If the call returns 5xx, then it will reset the timer on the Open state.

Provided By Author

If you still don’t understand, Martin Flower explains it well in his article.

Implementing Circuit Breaker

I will implement a simple circuit breaker as a wrapper that has 2 states — Open and Closed. You can also use Half Open as a state to do some more optimization in your circuit breaker wrapper, such as testing external service in percentage during the Half-Open condition to see if the state will go back to a Close state or not. However, for simplicity, the Half-Open state in the code below will be a function to see whether or not we will switch from Open to Closed state.

I will use the Cats effect, a famous Scala that adopts many functional libraries, with the Cats effect to write the circuit breaker.

First, before we start, we need to define the interface for what this wrapper can do. We also need to determine the three-state and the configuration for creating this Circuit Breaker.

We have a Status of Open and Close. Open will carry the timestamp's duration that it remains open and the error message of the reason the breaker set it open in the first place. Then, the Close holds the attempted error counter.

We also have CircuitBreakerConfig to set the configuration for the maximum number of retry the service can do before it switches to the Open state. When the status turns from Close to Open, it will construct an openTimestamp based on the current time + offsetTimeout.

Then, we want the client to run any of their code with the protect function. Something like this:

Keeping State in Concurrency Environment

Luckily, the Cats effect library concurrent primitive Ref is a pure atomic reference that helps you modify the value in a concurrent environment. It is the same as java atomic reference, except Ref uses tagless final abstraction and always contains value inside.

Ref is suitable in this kind of environment because the circuit breaker may be shared with multiple applications. Something like this:

But if you want to use a separate circuit breaker on two separate services, just don’t let both services shared the circuitBreaker together like this:

If you are interested in learning more about Ref, check out the Cats Effect library here.

Boilerplate Code

We create a boilerplate to implement the CircuitBreaker interface that we defined at the beginning of this article. Then, we can start implementing the executeOpenState and executeCloseState.

Creating Closed Status

  • check if the error count is over the threshold — if it is over the threshold set the state to Open with the currentTimestamp + offsetTimeout
  • increment the error count
  • raised that error and returned it back to the caller

The Open State

  • Check if the current time has passed
  • If the currentTime > openTimestamp, we go to a Half-Open state.
  • if the currentTime < openTimestamp, we will immediately throw a rejection failure response to the caller.

The Half Open State

Therefore, inside the executeHalfOpenBreaker, we simply call the executeCloseBreaker again to check if the execution succeeded. If it succeeded, we change the state back to Closed with an error count of 0. If it failed, the executeCloseBreaker would handle the case of failure.

That’s it! I hope you learn something from this article. You can wire any function onto the circuit breaker method to have that smart-ness by creating this. To simply illustrate this, let’s create 3 services that run concurrently, changing the resource ref:

In this example, we spawn 4 fiber, which is the equivalent of thread, and wire them with circuitBreaker. We set the config for the circuitBreaker to have a max number of retry of 3. Therefore, 3 failures call will set the status of the state to Open. The subsequent call will then failed. However, after waiting for 400 milliseconds, the last line (15), circuitBreaker.protect(callSuccess) will set the state to Half Open. Once the call succeeded, it will assess the condition to Close again.

I hope you learn something about circuit breaker and their internal implementation. Tose the full code, you can check out the GitHub link here.

Finally, some great resources about circuit breakers and other articles inspired me to implement this circuit breaker. Check out the resources below!

Originally published at https://edward-huang.com.

Javarevisited

Medium’s largest Java publication, followed by 10000+ programmers. Follow to join our community.

Thanks to Anupam Chugh

Sign up for Javarevisited Newsletter

By Javarevisited

Collection of best Java articles, tutorials, courses, books, and resources from Javarevisite and its authors, Java Experts and many more.  Take a look.

By signing up, you will create a Medium account if you don’t already have one. Review our Privacy Policy for more information about our privacy practices.

Check your inbox
Medium sent you an email at to complete your subscription.

Edward Huang

Written by

Software Engineer. Sharing my notes, thoughts, and document my journey in technology, functional programming, and careers in tech | https://www.Edward-Huang.com

Javarevisited

A humble place to learn Java and Programming better.

Edward Huang

Written by

Software Engineer. Sharing my notes, thoughts, and document my journey in technology, functional programming, and careers in tech | https://www.Edward-Huang.com

Javarevisited

A humble place to learn Java and Programming better.

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store