That’s a very good question. You are right that the rate limiter we open sourced runs on a single server. What we have here is a simple, high performance database, by which replication is not inherently supported.
There are many things you can do to mitigate this problem. As mentioned in the post, because we use Kubernetes and persistent disks to run our database cluster, it’s very easy to try so of the setups:
- Double write. Write to two replicas but allow one to fail. When reading the results, take the minimum of the two.
- Micro-partitioning. Run many small partitions of the rate limiter database, as it’s really easy to shard by the key. When you lose one partition, you only lose a small percentage of your db capacity for a few seconds until Kubernetes replace it.
These are just a few ideas. We may provide some solutions out of the box as we open source more projects in the future.