Rate Limiters

What is Rate Limiter?

Rate limiting is a strategy for limiting network traffic. It helps to control heavy traffic on your system and it prevents the system from crashing.

How does Rate Limiter work?

Just assume we have a system for the Department of Immigration & Emigration. This system gets heavy traffic usually. but a client sends a flood of traffic for your system to crash the system. The plan had to scale and it created multiple instances to manage traffic. also, the Load balancer distributes traffic for each instance. This is ok but this is unwanted traffic made by a client. It will make your bill increase and at one point the system will get crash.

As a solution, we can implement a Rate limiter in our system. We can configure a rate limiter with a number of requests allowing it to hit the system. As an example, we can configure the rate limiter to allow 10 requests for a second.

1000 miliseconds = 1 second

Just assume the system received 10 requests within 900 milliseconds. and now the rate limiter also allows going those requests to the system. The 11th request was also received by the system within that second. Then rate limit or check configuration and confirm that within a second rate limiter can allow only 10 requests and already 10 requests allowed. Then it is not allowed to pass the 11th request to the system. The rate limiter sends a response with a 429 status code to the client informing that too many requests. So as a client you have to wait until the next second to request because every 1 second it gets reset to allow 10 requests to pass the system.

We can set up rate limiters in a proxy. It is the most successful way to implement a rate limiter. If you try to implement a rate limiter with the service, it will be tightly coupled with the service and it will be huge when the service gets scaled.

Why Rate Limiter?

1. Preventing Resource Starvation: The most common reason for rate limiting is to improve the availability of API-based services by avoiding resource starvation. Load-based denial of service (DoS) attacks can be prevented if rate limiting is applied. Other users are not starved even when one user bombards the API with loads of requests.
2. Security: Rate limiting prevents brute forcing of security-intensive functionalities like login, promo code, etc. The number of requests to these features is limited on a user level so brute-force algorithms don’t work in these scenarios.
3. Preventing Operational Costs: In the case of auto-scaling resources on a pay-per-use model, Rate Limiting helps in controlling operational costs by putting a virtual cap on the scaling of resources. Resources might scale out of proportion leading to exponential bills if rate limiting is not employed.

Thank you!