Spring Cloud Microservices — Part 6— Implementing Retry with Resilience4j

Photo by David Pupaza on Unsplash

Introduction

To see the complete list of Spring Cloud Microservices tutorial series, you can check this link.

In this part of the tutorial series, we will discuss about how retry can be configured for service-to-service calls using Resilience4j library. You can check the previous tutorial about Circuit Breakers that will be helpful to move forward with this tutorial.

Let’s say that we have 2 services called Service A and Service B, and when Service A is called, it calls Service B internally. At some point there might be some network issues or service load that will make requests to Service B timeout. When such issues arise, we might want to make more than one call attempt to Service B in case of failures. This will give us another chance to get the successful response rather than receiving an error message. Retry is a good way to have your application call target methods/services automatically.

Application Setup

1- Add the following dependency to pom.xml:

<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-circuitbreaker-reactor-resilience4j</artifactId>
</dependency>
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-aop</artifactId>
</dependency>

spring-boot-starter-actuator dependency is used to give information about the application health, metrics etc. In this project we will use it to get information about the Retry status and events.

2- Configure application.yml as follows:

resilience4j.retry:
  configs:
    default:
      registerHealthIndicator: true
      maxAttempts: 3
      waitDuration: 1s
      enableExponentialBackoff: true
      exponentialBackoffMultiplier: 2
  instances:
    order-service:
      baseConfig: default
      ignoreExceptions:
        - io.github.resilience4j.circuitbreaker.CallNotPermittedException
management.endpoint.health.show-details: always
management.endpoints.web.exposure.include: health,info,circuitbreakers,circuitbreakerevents,retries,retryevents
management.health.retries.enabled: true

You can find the descriptions for all Circuit Breaker parameters here. I will describe a few of them here:

• resilience4j.circuitbreaker.configs.*.registerHealthIndicator parameter is used to enable health indicators for Circuit Breaker. In addition to that parameter, management.health.circuitbreakers.enabled value should also be set to true.
• resilience4j.circuitbreaker.instances.*.ignoreExceptions parameter is used to cancel retrying when these exceptions are received during method call.
• management.endpoint.health.show-details parameter will enable http://<service-url>/actuator/health URL to display the details of the health status. See the example output below:

{
  "status": "UP",
  "components": {
    "circuitBreakers": {
      "status": "UP",
      "details": {
        "order-service": {
          "status": "UP",
          "details": {
            "failureRate": "-1.0%",
            "failureRateThreshold": "100.0%",
            "slowCallRate": "-1.0%",
            "slowCallRateThreshold": "100.0%",
            "bufferedCalls": 0,
            "slowCalls": 0,
            "slowFailedCalls": 0,
            "failedCalls": 0,
            "notPermittedCalls": 0,
            "state": "CLOSED"
          }
        }
      }
    },
    ...
  }
}

• management.endpoints.web.exposure.include parameter determines which actuator endpoints will be enabled for access. In the above example we enable health, info, circuitbreakers, circuitbreakerevents, retries and retryevents endpoints which will allow us to access http://<service-url>/actuator/health, http://<service-url>/actuator/info, http://<service-url>/actuator/circuitbreakers and http://<service-url>/actuator/circuitbreakerevents endpoints.
• The endpoint /actuator/retries will display the available circuit breakers like the following:

{
  "retries": [
    "order-service"
  ]
}

3- Annotate your methods using @Retry annotation to enable Retry. You can use this annotation either with concrete classes or even with OpenFeign client proxies as follows:

import com.oardic.userservice.model.Order;
import io.github.resilience4j.circuitbreaker.CallNotPermittedException;
import io.github.resilience4j.circuitbreaker.annotation.CircuitBreaker;
import io.github.resilience4j.retry.annotation.Retry;
import org.springframework.cloud.openfeign.FeignClient;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;

import java.util.List;

@FeignClient("${services.orderService.name}")
public interface OrderServiceFeignProxy {

   @GetMapping("/orders/user/{userId}")
   @Retry(name = "order-service", fallbackMethod = "getUserOrdersFallbackForRetry")
   @CircuitBreaker(name = "order-service", fallbackMethod = "getUserOrdersFallbackForCircuitBreaker")
   List<Order> getUserOrders(@PathVariable long userId);

   default List<Order> getUserOrdersFallbackForRetry(Exception e) {
      return List.of(Order.builder().id(-2L).build());
   }

   default List<Order> getUserOrdersFallbackForCircuitBreaker(CallNotPermittedException e) {
      return List.of(Order.builder().id(-1L).build());
   }
}

• name attribute must point to an existing resilience4j.retry.instances.<name> value in application.yml file.
• fallbackMethod is an optional attribute that points to a method to be called, if all retries fail.

You can combine @Retry and @CircuitBreaker annotations to work together as seen in the above example. In this case @CircuitBreaker will be called before @Retry. All Resilience4j annotations will be called in a specific order; Retry ( CircuitBreaker ( RateLimiter ( TimeLimiter ( Bulkhead ( Function ) ) ) ) ) from inside to outside. So, @Bulkhead will be called first and @Retry will be called last.

Monitoring Retry Events

The endpoint /actuator/retryevents will display the list of Circuit Breaker events. In our example project, we are using Retry with User Service, so we can see the events via http://localhost:8080/actuator/retryevents:

{
  "retryEvents": [
    {
      "retryName": "order-service",
      "type": "RETRY",
      "creationTime": "2022-10-10T16:44:47.745507100+03:00[Asia/Istanbul]",
      "errorMessage": "feign.RetryableException: Connection refused: connect executing GET http://order-service/orders/user/1",
      "numberOfAttempts": 1
    },
    {
      "retryName": "order-service",
      "type": "RETRY",
      "creationTime": "2022-10-10T16:44:52.843516600+03:00[Asia/Istanbul]",
      "errorMessage": "feign.RetryableException: Connection refused: connect executing GET http://order-service/orders/user/1",
      "numberOfAttempts": 2
    },
    {
      "retryName": "order-service",
      "type": "ERROR",
      "creationTime": "2022-10-10T16:44:58.938701+03:00[Asia/Istanbul]",
      "errorMessage": "feign.RetryableException: Connection refused: connect executing GET http://order-service/orders/user/1",
      "numberOfAttempts": 3
    }
  ]
}

Conclusion

@Retry annotation provided by Resilience4j library lets us easily retry failed requests. You can combine it with other Resilience4j annotations to provide additional features to your methods.

Source Code

You can download the complete source code of this tutorial series from this link.

References

https://resilience4j.readme.io/docs/retry

https://resilience4j.readme.io/docs/getting-started-3