An introduction to the django-carrot task backend

django-carrot is a lightweight task queue for Django projects with an emphasis on very easy setup and task traceability. It is intended as a simpler alternative to the celery project. This tutorial will explain how to add a django-carrot task backend to your project in a matter of minutes.

Setting up RabbitMQ

django-carrot uses the RabbitMQ message broker to queue and execute tasks. It can be installed and started using brew:

brew install rabbitmq
brew services start rabbitmq

You should now be able to see the RabbitMQ monitor at http://localhost:15672/#/:

Installing django-carrot

django-carrot can be installed with pip:

pip install django-carrot

Using django-carrot in your project

If you don’t already have an existing Django project, create one in the normal way:

django-admin.py startproject myproject

If you already have a Django project, you can add django-carrot to the installed apps in your settings.py file:

INSTALLED_APPS = [
...
'carrot',
]

django-carrot stores execution information for all tasks in the Django project’s database. You’ll need to create and apply migrations in order for it to work

python manage.py makemigrations carrot
python manage.py migrate

You can now start the django-carrot service as follows:

python manage.py carrot_daemon start --logfile carrot.log
The default value of the logfile parameter is /var/log/carrot.log. You will need to change this, as above, if this path does not exist on your machine or you do not have access to it

Go back to the RabbitMQ monitor and look at the queues tab. You should be able to see that a queue called default has been created.

Stop the service for now with the following command:

python manage.py carrot_daemon stop

Creating tasks

Now that you have a queue set up, you can publish tasks to it. django-carrot can execute any Python function as an asynchronous task. For example, this simple function multiplies two numbers together and returns a string with the result

def multiply(a, b):
return '%i multiplied by %i is %i' % (a, b, a * b)

Tasks must first be published to your RabbitMQ queue. This is done using django-carrot’s publish_task utility:

from carrot.utilities import publish_task
publish_task(multiply, 2, 2)

If you go back to RabbitMQ, you should be able to see the task that you just published by clicking on the default queue and clicking the Get Message(s) button

Tasks will sit in the queue until the django-carrot service consumes and executes them. Turn the django-carrot service back on again, and you should see the task disappear from the queue, and the message count in RabbitMQ will return to zero:

python manage.py carrot_daemon start --logfile carrot.log

Monitoring tasks

Although the task has been completed, we haven’t yet seen any execution information about it. Fortunately, django-carrot provides its own monitor which stores detailed information about all tasks published, completed, scheduled and failed.

To enable it, go to your project’s urls.py file, and add it as follows:

urlpatterns = [
...
url(r'^carrot/', include('carrot.urls')),
]

Restart your Django server and navigate to http://localhost:8000/carrot/. You’ll be able to see the task that was just executed under the Completed tab:

Click on the task name to see more details, and the output

More options

Logging

Logging can be used to aid debugging of tasks in the carrot monitor as follows:

from carrot.utilities import publish_message
import logging
logger = logging.getLogger('carrot')
def my_task(*args, **kwargs):
logger.debug('hello world')
logger.info('hello world')
logger.warning('hello world')
logger.error('hello world')
logger.critical('hello world')
publish_task(my_task)

This would appear in the monitor as follows:

Note that the debug message is not displayed, as the log level drop down is set to INFO

Concurrency

By default, the carrot service creates one consumer and one queue. Multiple queues and consumers can be created by adding the CARROT attribute to the settings file:

from carrot import DEFAULT_BROKER
CARROT = {    
'default_broker': DEFAULT_BROKER,
'queues': [
{name: 'queue1', 'vhost': DEFAULT_BROKER'},
{name: 'queue2', 'vhost': DEFAULT_BROKER', 'concurrency': 10}
]
}

Scheduled tasks

Tasks can also be scheduled to run at intervals via the monitor

For further reading, refer to the django-carrot docs at http://www.django-carrot.com/