Rate limiting, brute force and DDoS attacks protection in Node.js

The best way of protection is configuring web server, CDN, load balancer or any other entry point to your node.js application.

However, it doesn’t fit requirements sometimes.

Recently, I had a task to protect distributed, high traffic Express.js application by some sophisticated rules. So I needed fast and reliable solution.

I’ll tell you about my experience with requirements, benchmarks and conclusions. Fasten your seatbelts! :)

tl;dr rate-limiter-flexible package helps

Requirements

1. fast. we don’t want rate limiting takes 100–200ms on each request, right?
2. reliable. Redis or any other in-memory store usually used as limits store for distributed applications, but any store may down sometimes. The solution must provide some insurance. We don’t want our service 30 seconds down time just because of some package.
3. flexible. I don’t want a package, which makes decisions for me and sends some response headers. I’d like to change behaviour for each case and be sure nothing happens invisibly.
4. block strategy. Even in-memory stores like Redis and Memcache have some limits of maximum operations per second. So if DDoS attack is really powerful and makes over 50k-100k requests per second, it’ll significantly slow down all requests. Or even worse — make it down completely.
5. have to make the difference between brute force and real users. Sometimes real clients and third-party applications may look like brute force. Our application must make the difference between real users which send wrong requests by some reason and malicious brute force requests
6. cut off load picks Third-party applications usually configure some activity on certain time during a day. It may result to instant growing of traffic, for example, from 100 to 20k requests per second. The solution has to additionally provide some delaying approach. It’s better to delay a portion of requests for 1–2 seconds, if a database or some part of an application can’t handle them all at once. Queuing is better solution though, but I didn’t want to bring another software to the application
7. modern Native promises and async/await syntax
8. different rules for different endpoints

Solution

I’ve searched open-source solution on Github. There are many good libraries, but they don’t fit all my requirements.

I came up to developing the new package rate-limiter-flexible

I started from Redis support as it was the requirement. Also, there are in-memory, cluster with IPC and MongoDb implemented limiters now.

RateLimiterRedis

It supports both redis and ioredis the most famous Redis node.js clients.

Redis client must be created with offline queue switched off. It is important, because it must work with real data.

Also, as a rule, we don’t need persistence, so it’s worth to disable Redis persistence with configuration save “” and appendonly no. It makes Redis 2–3% faster and prevents doing heavy disk I/O. You can read about it more in official docs

Here is basic example

const { RateLimiterRedis } = require('rate-limiter-flexible');
const Redis = require('ioredis');
// It is recommended to process Redis errors and setup some reconnection strategy
const redisClient = new Redis({
  options: {
    enableOfflineQueue: false
  }
});

const opts = {
  storeClient: redisClient,
  points: 10, // Number of points
  duration: 1, // Per second(s)
};

const rateLimiterRedis = new RateLimiterRedis(opts);

rateLimiterRedis.consume(remoteAddress)
    .then((rateLimiterRes) => {
      // ... Some app logic here ...
    })
    .catch((rejRes) => {
      if (rejRes instanceof Error) {
        // Some Redis error
      } else {
        const secs = Math.round(rejRes.msBeforeNext / 1000) || 1;
        res.set('Retry-After', String(secs));
        res.status(429).send('Too Many Requests');
      }
    });

Express middleware example

const rateLimiterMiddleware = (req, res, next) => {
  rateLimiter.consume(req.connection.remoteAddress)
    .then(() => {
      next();
    })
    .catch((rejRes) => {
      res.status(429).send('Too Many Requests');
    });
};

Let’s go through requirements

Fast

There are two groups of algorithms to limit requests using Redis: with fixed window and with rolling window. Here is a good article about rolling window approach. However, it works 10–20 times slower on high traffic than fixed window.

Here is average latency comparison of rate-limiter-flexible RateLimiterRedis with other node.js packages with rolling window.

A simple Express.js 4.x endpoint is limited by different libraries with the same rules. Application is launched in node:latest and redis:alpine Docker containers by PM2 with 4 workers.

Tested bybombardier -c 1000 -l -d 30s -r 2000 -t 1s

• 1000 concurrent requests
• test duration is 30 seconds
• not more than 2000 req/sec

Results:

• rate-limiter-flexible 11ms
• rolling-rate-limiter 282ms
• ratelimiter 219ms
• redis-rolling-limit 148ms

More info about this comparative benchmark here

As you can see, RateLimiterRedis at least 14 times faster on high traffic.

11 ms is good enough for Express.js endpoint, agree?

Reliable

Rate-limiter-flexible has insurance strategy. It comes with RateLimiterMemory and RateLimiterCluster from the box, which limits requests in current process memory or using IPC (inter-process communication) for a cluster on one server. Both don’t require any additional software. It helps if Redis is down.

const opts = {
  redis: redisClient,
  points: 100, // Number of points
  duration: 1, // Per second(s)
  insuranceLimiter: new RateLimiterMemory(
    {
      points: 20, // 20 is fair if you have 5 workers and 1 cluster
      duration: 1,
    })
};

insuranceLimiter can be any rate limiter from rate-limiter-flexible package. Even the second instance of Redis.

Flexible

Rate-limiter-flexible doesn’t define any internal logic for different user roles, headers or endpoints.

Your application decides how many points can be consumed like:

let consumePoints = role === 'guest' ? 2 : 1;
rateLimiter.consume(userId, consumePoints) ...

In this case logged in user can make twice more requests per same duration.

Both Promise resolve and reject returns RateLimiterRes object with attributes:

RateLimiterRes = {
    msBeforeNext: 250, // next action can be done after
    remainingPoints: 0, // in current duration 
    consumedPoints: 5, // in current duration 
    isFirstInDuration: false, // action is first in current duration 
}

Decide what to do with it or do nothing. It is up to you!

Block strategy

Block strategy is against DDoS attacks. Redis is quite fast. It can process over 100k requests per second. However, performance still depends on amount of requests per second.

We don’t want latency to become 3, 5 or more seconds. RateLimiterRedis and all other limiters from rate-limiter-flexible, which work with some store, provide a block strategy to avoid too many requests during DDoS attack.

const opts = {
  storeClient: redisClient,
  inmemoryBlockOnConsumed: 200, // If 200 points consumed
  inmemoryBlockDuration: 30, // block for 30 seconds
};

It blocks in current process memory. Yes, not too accurate in distributed environment, but we don’t need really accurate thing for this particular case. If DDoS requests go from some clients with rate like 10–20k per second, it is obvious they reach block limits on all workers instantly. And Redis, MongoDb or any other store is safe.

There is an example of combining Block Strategy with blockDuration option to make blocks consistent over all Node.js processes.

Difference between brute force and real users

Usually brute force attack result to tries getting nonexistent data by id or some unique path. We can use it and consider. There is useful penalty method.

rateLimiter.consume(remoteAddress)
    .then((rateLimiterRes) => {
      const data = getData(id);
      if (data === null) { // possible brute force request
        rateLimiter.penalty(remoteAddress, 5);
      }     
    })
    .catch((rejRes) => {
      
    })

This way real users accidentally made wrong request are not blocked, but brute forcing clients are blocked faster.

Cut off load picks

rate-limiter-flexible execEvenly option allows to delay actions and execute them evenly over duration. First action in duration is executed without delay. All next allowed actions in current duration are delayed by formula msBeforeDurationEnd / (remainingPoints + 2) However, it isn't recommended to use it for long durations, as it may delay action for too long.

const opts = {
  points: 3, // Number of points
  duration: 3, // Per second(s)
  execEvenly: true,
};

It is example of limiting rate by 3 points per 3 seconds. User may consume all 3 points during the first second and then wait 2 seconds until next duration. Such approach results to possible load pick. If execEvenly set up to true , rate limiter delays the second and the third request using setTimeout

Modern

rate-limiter-flexible backed on native Promises. So async/await syntax can be easily used

try {
  await rateLimiter.consume(req.connection.remoteAddress);
  next();
} catch (rejRes) {
  res.status(429).send('Too Many Requests');
}

Different rules for different endpoints

You just need to set different keyPrefix option

const rl1 = new RateLimiterRedis(
{
  keyPrefix: 'limiter1',
  points: 3,
  duration: 3,
  execEvenly: true
});
const rl2 = new RateLimiterRedis(
{
  keyPrefix: 'limiter2',
  points: 1,
  duration: 5,
});

More advantages

1. NO production dependencies
2. covered by tests

Here we are! Finally.

Bonus

rate-limiter-flexible comes up with implemented RateLimiterRedis, RateLimiterMongo, RateLimiterCluster and RateLimiterMemory classes.

2019 update: rate-limiter-flexible supports PM2, PostgreSQL, MySQL and Memcached now. There is also Black-and-White wrapper by list or function. TypeScript declaration bundled.

Get more examples on Wiki

Benchmark

Average latency of pure NodeJS endpoint limited by (all set up on one server):

1. Memory   0.34 ms
2. Cluster  0.69 ms
3. Redis    2.45 ms
4. Mongo    4.75 ms

Expected result, right? Cluster is useful when your application is just launched on one server, we don’t need any additional software in this case.

MongoDb is quite famous. Many applications process small amount requests per second. In this case, the same instance can be used for storing data and limiting. No harm at all.

By the way, it’s recommended to launch the second instance of MongoDb with persistence off. Just launch it with —- syncdelay 0 —- nojournal —- wiredTigerCacheSizeGB 0.25 No snapshots, no journaling and 256Mb of memory. It becomes in-memory storage roughly twice slower than Redis, but it is enough in many cases.

Enterprise MongoDb provides in-memory engine by default. Should be good too, if persistence isn’t required.

May be MongoDb is your choice to limit requests, it is fast enough. And usually already in environment Read more about it here

Future

There are many good and modern in-memory stores and databases like Tarantool, Aerospike, LevelDb, RocksDb, etc.

Please, post a comment with suggestions, which you think is better than Redis for rate limiting.

Feel free to participate and extend rate-limiter-flexible .

Thank you for reading and happy coding!