How to create a dead letter queue in Celery + RabbitMQ

Recently, I started learning how to run async tasks in Celery, which has been used in our company product. One interesting requirement is how to create a dead letter queue in Celery?

RabbitMQ has already supported arguments, x-dead-letter-exchange and x-dead-letter-routing-key, when creating a new queue.

I have made a simple example to create a dead letter queue:

You need a rabbitmq as a broker and redis as a backend. Run docker-compose up -d to start containers.

Just run:

celery -A tasks worker --loglevel=info
# open ipython
from tasks import add, div
add.delay(1,2)
div.delay(2,1)
div.delay(2,0) # throws ZeroDivisionError

Reject has to be used with acks_late together, because by default the worker will acknowledge to remove the message from the queue. If acks_late is set, an acknowledgement is sent after the result of task is returned. When acks_late is enabled, the worker can reject the task that will be redelivered to a dead letter queue.

Note:

• Originally, I had the same that if we don’t include deadletter queue into app.conf.task_queues and start workers at first time, deadletter queue will not be created.
• According to this question, K.P. proposed a solution to create deadletter queue which works very well. So I modified my example as well.