Rest Template & Connection Pool
2 min readDec 30, 2020
Baeldung's article about HTTP connection management explains how to configure an HTTP connection pool for the Spring Boot Rest Template.
The following is a tale about what happens when you didn't read it thoroughly enough!The context
Once upon a time, I had a Spring Boot consumer-facing microservice and a backend microservice. The consumer-facing microservice makes a request to the backend microservice for each request received.
In order to load test the consumer-facing microservice, the backend service is replaced with a stub. Stub behavior is simple, it waits 500ms before replying.If you want to load test a service, it's crucial that its dependencies are stubbed with a realistic delay.
Let's see later why...Load testing can be done with many tools. I personally like Artillery.IO for it's simple for powerful enough for most of my use cases.
The Drama
When I loaded testing, my consumer-facing microservice crumbled quickly at around 10 requests per second (10 RPS)!The microservice under test and the stub are both Spring Boot applications which embedded a tomcat server.
At first sight, the stub may be pointed out as the performance bottleneck but by default, tomcat has a maximum of 200 threads which is by far more than needed here.The bad guy is found
This configuration is inline with another Baeldung's article about rest template builder. It seems nice and clean but it hides a default PoolingHttpClientConnectionManager with the defaultMaxPerRoute set to 5.
What does this default max per route means? It means that only 5 simultaneous HTTP connections to the same host will be possible.
In my situation, the backend takes 500ms to answer, so 5 simultaneous connections means a hard limit to 10 RPS!
By reversing the equation from this article from thread pool sizing,
we see that the obtained 10 RPS could easily be predicted.λ = L / W
λ – long-term average arrival rate (RPS)
L – the number of requests processed simultaneously
W – the average time to handle the request (latency)
Conclusion
- Avoid default configuration for connection pool.
- Know your dependency latency as it directly affects your resources (connections, threads) consumption.
- Perform load testing.A step further
Another drama with the https self-signed configurationIf you followed another excellent Baeldung's article about https self-signed certificate in the context of a Spring Boot App,
you ended up with the default PoolingHttpClientConnectionManager with the defaultMaxPerRoute configured to 2.
The same https self-signed certificated configuration but with an HTTP pooled connection manager configured.
Of course, the max connection per route should be tailored according to your needs.