[PoC6] serving 2.4 millions/minute REST web service requests hosted on Netty, proxied through NGINX

A determined woman works with a hardened boxing trainer to become a professional.

Clint’s directing is better than his acting IMO. He is like a good taste of wine, getting better and better every year. When I look back I see that it’s been 10 years now but the title Million Dollar Baby doesn’t get old.

To serve millions of REST web service requests you don’t need to have million dollars hardware. But a determined developer can cope with this situation with the help of the cloud. My last try-out already hit the number 20k/sec so I thought having 2 Netty servers might boost the output numbers in total. So my first try-out was as stated in the following diagram. I used the AMI ami-a8221fb5 and the instance type m4.xlarge for creating all those 4 instances.

http-requester creates the request load on NGINX and NGINX delegates them onto servers that contain the implementation netty-rest-simple. With this setup I was only to get 22k/sec at the saturation point.

It occurred to me that I have a bottleneck on the network bandwidth since XLarge could only fire upto 1Gbps. So I should definitely try the 10Gbit goodness. I swapped the instance C with a 10xLarge machine as given in the following diagram:

Well it was totally fresh air. My http-requester parameters were same with the previous PoCs. Here are the results:

1642 req/sec
4510 req/sec
5433 req/sec
5371 req/sec
5548 req/sec
6384 req/sec
7192 req/sec
7458 req/sec
7800 req/sec
8119 req/sec
8243 req/sec
8618 req/sec
8477 req/sec
8173 req/sec
8584 req/sec
8624 req/sec
8429 req/sec
8620 req/sec
8724 req/sec
9115 req/sec
9201 req/sec
9695 req/sec
9960 req/sec
11798 req/sec
11826 req/sec
12943 req/sec
14098 req/sec
15178 req/sec
15556 req/sec
16747 req/sec
17021 req/sec
17552 req/sec
18130 req/sec
18655 req/sec
19164 req/sec
19318 req/sec
20928 req/sec
20534 req/sec
20257 req/sec
20884 req/sec
21485 req/sec
21574 req/sec
24804 req/sec
25615 req/sec
26858 req/sec
26904 req/sec
26878 req/sec
28151 req/sec
27252 req/sec
28138 req/sec
29118 req/sec
28091 req/sec
28929 req/sec
28278 req/sec
27840 req/sec
28654 req/sec
28247 req/sec
28463 req/sec
29114 req/sec
28960 req/sec
29695 req/sec
28786 req/sec
29156 req/sec
29098 req/sec
29729 req/sec
24757 req/sec
33989 req/sec
36059 req/sec
35682 req/sec
36801 req/sec
35612 req/sec
37790 req/sec
38559 req/sec
38336 req/sec
36688 req/sec
38867 req/sec
39353 req/sec
39388 req/sec
38713 req/sec
39318 req/sec
39188 req/sec
38741 req/sec
38360 req/sec
38638 req/sec
39620 req/sec

Numbers are saturated at 39k/sec, which sums up to over 2 million requests/min. yay!