Increase the Resilience Score of your .NET application
Strategies for Building Fault-Tolerant Systems
What is resiliency in the backend context?
Resiliency in the backend context refers to an application’s ability to recover from transient failures and continue functioning normally. In .NET programming, this can be achieved by designing applications that can gracefully handle failures and recover quickly.
Why do we need this pattern on the backend side?
The need for this pattern on the backend side arises because networks are inherently unpredictable. Latency, connectivity issues, upgrading the apps, the container being restarted, the requests being rate limited, the server outages and many more can all cause disruptions in communication between applications. Therefore, it becomes crucial to incorporate resiliency into the HTTP Client to create applications that can withstand and recover from failures.
Resilience strategies:
- Rate limiter
- Total request timeout
- Retry
- Circuit breaker
- Attempt timeout
- Hedging
Rate limiter
The rate limiter pipeline limits the maximum number of concurrent requests being sent to the dependency. There are multiple different rate-limiting algorithms to control the flow of requests. We’ll go over 4 of them that will be provided in .NET 6.
The concurrency limiter limits how many concurrent requests can access a resource. If your limit is 10, then 10 requests can access a resource at once and the 11th request will not be allowed.
Another rate limiting is the token bucket algorithm. This approach generates tokens at a fixed rate, which are then consumed with each API request. Once the tokens are depleted, further requests are denied until new tokens are generated.
Fixed Window Rate Limiting works by dividing time into fixed windows and allowing a certain number of requests in each window. For example, if we have a limit of 100 requests per hour, and our window starts at 2:00, we can make 100 requests between 2:00 and 3:00.
The Sliding Window Algorithm is a time-based method used to track and control the rate at which requests or operations can be made within a specific time window. It’s a dynamic system that adapts to changing traffic patterns, making it an effective tool for rate limiting in various applications.
Retry
The retry pipeline retries the request in case the dependency is slow or returns a transient error.
Circuit breaker
The circuit breaker blocks the execution if too many direct failures or timeouts are detected.
Attempt timeout
The attempt timeout pipeline limits each request attempt duration and throws if it’s exceeded.
Total request timeout
The total request timeout pipeline applies an overall timeout to the execution, ensuring that the request, including retry attempts, doesn’t exceed the configured limit.
Hedging
The hedging strategy executes the requests against multiple endpoints in case the dependency is slow or returns a transient error. Routing is optional, by default it just hedges the URL provided by the original “HttpRequestMessage”. This strategy is really useful when you intend to route a percentage of the requests to a different endpoint.
How to implement resiliency in .NET apps?
The configuration is really easy, even if you decide to use resiliency after some time when the development started. The configuration should be done in the Program class and the services that make http requests should inject a typed HTTP client.
To begin, you need to install two NuGet packages: Microsoft.Extensions.Resilience and Microsoft.Extensions.Http.Resilience.
After that, you have to replace your way of using HTTP client with a typed client in your services. Then you have to configure the typed http client for each service that you have. This is very powerful for resiliency because you can configure the strategies for each service based on where it makes HTTP calls.
var httpClientBuilder = services.AddHttpClient<MyAwesomeClient>(
configureClient: static client =>
{
client.BaseAddress = new("https://very.awesome.typicode.com");
});private readonly HttpClient _httpClient;
public MyAwesomeClient(HttpClient httpClient)
{
_httpClient = httpClient;
}Once the HTTP client is injected and configured the next step is to register the resilience handler and to do that we have 3 options:
- Standard Handler — this handler comes with default values for each strategy and it is enough to register it
httpClientBuilder.AddStandardResilienceHandler();- Hedging Handler — the hedging handler allows the http client to make requests on multiple base URLs and to apply the strategies on each of them according to the “weight” property that can be configured. As the documentation said this is optional and it also comes with default values for each strategy.
httpClientBuilder.AddStandardHedgingHandler(static (IRoutingStrategyBuilder builder) =>
{
// Hedging allows sending multiple concurrent requests
builder.ConfigureOrderedGroups(static options =>
{
options.Groups.Add(new UriEndpointGroup()
{
Endpoints =
{
// Imagine a scenario where 3% of the requests are
// sent to the experimental endpoint.
new() { Uri = new("https://example.net/api/experimental"), Weight = 3 },
new() { Uri = new("https://example.net/api/stable"), Weight = 97 }
}
});
});
});- Custom Handler — with his handler only the strategies that are added in the builder will be applied. For example, the retry strategy is configured in the builder which means that only the retry will be executed on fails. The next code covers the retry strategy with 5 retry attempts and the time between them is exponential (1s, 2s, 4s, 8s, 16s). Next, it is configured the circuit breaker, it allows failed requests only for “RequestTimeout” or “TooManyRequests” status codes, and the last one is the timeout strategy which will throw an error if the request has a response time of more than 5s.
httpClientBuilder.AddResilienceHandler(
"AwesomePipeline",
static builder =>
{
builder.AddRetry(new HttpRetryStrategyOptions
{
BackoffType = DelayBackoffType.Exponential,
MaxRetryAttempts = 5
});
builder.AddCircuitBreaker(new HttpCircuitBreakerStrategyOptions
{
SamplingDuration = TimeSpan.FromSeconds(10),
FailureRatio = 0.2,
MinimumThroughput = 3,
ShouldHandle = static args =>
{
return ValueTask.FromResult(args is
{
Outcome.Result.StatusCode:
HttpStatusCode.RequestTimeout or
HttpStatusCode.TooManyRequests
});
}
});
builder.AddTimeout(TimeSpan.FromSeconds(5));
});Conclusion
Embracing resiliency in .NET applications is not merely a best practice; it’s a strategic imperative for modern development. As we conclude our exploration into the world of building resilient .NET apps, one thing becomes clear: the digital landscape is rife with uncertainties and challenges. By incorporating resiliency patterns and strategies into your .NET development toolkit, you empower your applications to gracefully handle failures, adapt to changing conditions, and provide a seamless user experience.
In essence, resiliency is the cornerstone of robust, high-performance software. It’s not just about mitigating errors; it’s about thriving in the face of adversity. From circuit breakers to retries, graceful degradation to fault tolerance, each aspect contributes to the overall resilience of your .NET applications.
Special thanks to
and for their unwavering support and insightful feedback.