Hello, you’r totally right.
Luc Juggery

Well, I was hoping you’d tell me you’ve tried it and it works ;) It seems like this would need a redis cluster config (https://redis.io/topics/cluster-tutorial), however that comes with the caveat that it required host networking mode, which is not yet supported in docker-compose files v3.

We’re using redis with celery so MongoDB is not an option, unfortunately.

One thing I am going to try and see is if redis allows for an immediate ack to the node a work item is pulled from, if you can afford to do that. (We could.) You’d lose some guarantees but if that’s ok that might allow for simple scaling.

Like what you read? Give Sven A. Schmidt a round of applause.

From a quick cheer to a standing ovation, clap to show how much you enjoyed this story.