Don’t Overlook this Critical Spring Configuration; connectionRequestTimeout

image from pixabay

How many timeout configurations are you aware of? What about connectionRequestTimeout? If you haven’t heard of the connectionRequestTimeout yet, welcome! In this post, we’ll explore the critical importance of understanding connectionRequestTimeout in Spring and how to configure it properly.

Let me tell you about my experience. One of the external systems failed and caused a connection timeout, meaning my system couldn’t reach out and create a connection to it. I expected my API to take 5 seconds at maximum since I set connection timeout 5 seconds for it. However, many requests took more than 40 seconds in that situation.

Why it took more than 5 seconds? How will you approach this problem?

Well, the answer is in the title. I didn’t set a connectionRequestTimeout configuration for HttpClient.

Connection Pool and Performance in Spring

Before diving into the connectionRequestTimeout, We need to understand how external HTTP requests are made in Spring.

Spring uses connection instances to connect external components using HTTP. Spring uses a connection pool to reuse them. Creating a connection for an HTTP request is an expensive process. Below is the reason why it is expensive.

• DNS resolution: Before a connection can be established, the client must first resolve the domain name of the server to an IP address using the Domain Name System (DNS). DNS resolution can take time, especially if the client’s DNS cache is cold and the server’s domain name has not been cached.
• TCP handshake: After the IP address of the server has been determined, the client must then establish a TCP connection with the server. This involves a three-way handshake process, which can take time and add overhead to the request.
• SSL/TLS negotiation: If the server is using HTTPS, the client must also negotiate an SSL/TLS connection with the server before any data can be exchanged. This involves a complex handshake process that can add significant overhead to the request.

Using connection pool for HTTP requests can improve performance significantly. However, we need to configure it properly to prevent a disaster situation.

What is connectionRequestTimeout?

Now let’s see the problem again with the concept of the connection pool.

In this example, the B component is in a situation where it can’t establish an HTTP connection which causes the connection timeout in A. Please look closely at the above A’s arrow to the connection pool. A is waiting for yet another connection to establish an HTTP connection to B.

The software might fail in various ways unless you set proper timeout settings for waiting situations. In this example, the timeout you need is called the connectionRequestTimeout.

ConnectionRequestTimeout is a configuration parameter in Spring that determines how long a client will wait for a connection from the connection pool before timing out. This timeout value is used to prevent the client from waiting indefinitely for a connection that may not be available, and to free up resources in the connection pool when they are no longer needed.

OK… But What is the proper value for it?

The default value of the connectionRequestTimeout is -1, meaning it waits for connections from the connection pool indefinitely. Since we want to avoid system failure by external components outage, we need to set an explicit value for it.

Let’s imagine you set the value too short: 1 second. This could be a frequent failure if the system requires high latency because it is likely hungry for connection all the time. On the other hand, If the value is too long, like 10 mins, the system can easily fail with external failures.

Hence we need to investigate the request frequency pattern of the system. Let’s simplify an API response time.

API response time = connectionRequestTimeout + connectionTimeout + readTimeout

Collect API response time and see if many requests exceed the sum of connectionTimeout and readTimeout. If so, the system needs generous connectionRequestTimeout. Otherwise, set connectionRequestTimeout reasonably from 15 seconds to 30 seconds.

Coding Time :)

Here is an example code to set connectionRequestTimeout.

import org.springframework.http.client.HttpComponentsClientHttpRequestFactory;
import org.springframework.web.client.RestTemplate;

public class ConnectionRequestTimeoutExample {

    public static void main(String[] args) {
        RestTemplate restTemplate = new RestTemplate();
        HttpComponentsClientHttpRequestFactory requestFactory = new HttpComponentsClientHttpRequestFactory();
        // Set the connectionRequestTimeout value to 10 seconds
        requestFactory.setConnectionRequestTimeout(10000);
        restTemplate.setRequestFactory(requestFactory);
        ...
    }
}

In this example, we create a new RestTemplate and a new HttpComponentsClientHttpRequestFactory. We then set the connectionRequestTimeout value to 10 seconds using the setConnectionRequestTimeout() method, and we set the request factory for the RestTemplate using the setRequestFactory() method.

Conclusion

As a software engineer, I saw many outages in which the root cause was an improper setting in the configuration file. The connectionRequestTimeout is one of the crucial but easily forgotten configurations. Please remember to set the below three configurations for Spring.

• connectionRequestTimeout: wait time to get a connection from a connection pool.
• connectionTimeout: wait time to establish a connection to the external component.
• readTimeout: wait time to get the response from the external component.

Always thanks for reading :)