Improving Django performace

Some best practices I learned the hard way

Following this question from Quora where someone asked how to improve Django performance, I wrote an answer stating some points without making efforts to explain it better.

Redis

Cache everything you can. This is a rule, really. Cache your views, your database queries.

I personally prefer Redis everywhere, it is fast. And you can set it up based on your needs, but this is actually another story.

At the time of writing, you can use Redis on multiple layers:

• Query Caching: you can use something that does that for you like django-cacheops or johnny-cache. If you want something that works magically and that you don’t have to worry about use the second one.
• View Caching using django internal caching framework with django-redis-cache backend.
• Session storage, you can save your sessions directly in redis using django-redis-sessions
• Celery: if you use celery you can configure it to use Redis as your backend.

Gunicorn

When you will be ready for production, really, don’t look anything else. Just use gunicorn. It’s easy to setup, it’s easy to customize and its performance are perfectly comparable to uWSGI or Tornado. You just need a good setup:

• Use (n*2)+1 workers
• Use gevent

Also, if you will use gunicorn on Ubuntu, set Gunicorn with Supervisor, here it is a configuration for a 16 core web server:

[program:gunicorn]
command=unicorn -w 33 -b 127.0.0.1:8000 -k gevent app.wsgi
user=user
directory=/home/user/path_to_project/
stdout_logfile=/home/user/logs/gunicorn-stdout.log
stderr_logfile=/home/user/logs/gunicorn-stderr.log
autostart=true
autorestart=true
redirect_stderr=true
priority=990

Celery

I have already mentioned Celery, a distributed task queue with a solid integration with Django.

When using Django and you need some tasks that can be done asynchronously, just write your tasks in a python file and use the Django’s admin interface to schedule them with a crontab or an interval then run celerybeat to send them to the celeryd workers.

It’s as simple as that.

Debug, Debug and Django-Debug-Toolbar

Django Debug Toolbar is one of the best tool I have ever used when in developing mode. It tracks everything you need to look at, especially queries. You can use this information to know where your application is failing and where to improve.

Here is where I can log slow queries and use more “select_related” or “prefetch_related”: these two methods are performance boosters when you access foreign keys.