Things You Must Know About Django Admin As Your App Gets Bigger

Haki Benita
Aug 5, 2016 · 8 min read

For a better reading experience, check out this article on my website.


The Django admin is a very powerful tool. We use it for day to day operations, browsing data and support. As we grew some of our projects from zero to 100K+ users we started experiencing some of Django’s admin pain points — long response times and heavy load on the database.

In this short article I am going to share some simple techniques we use in our projects to make the Django admin behave as apps grow in size and complexity.

We use Django 1.8, Python 3.4 and PostgreSQL 9.4. The code samples are for Python 3.4 but they can be easily modified to work on 2.7 and other Django versions.


Before We Start

The List View

The main components of Django admin list view:

Example admin list view including some of the components discussed in the article

Logging

Most of Django’s work is performing SQL queries so our main focus will be on minimizing the amount of queries. To keep track of query execution you can use one of the following:

  • django-debug-toolbar — Very nice utility that adds a little panel on the side of the screen with a list of SQL queries executed and other useful metrics.
  • If you don’t like dependencies (like us) you can log SQL queries to the console by adding the following logger in settings.py:

The N+1 Problem

The N+1 problem is a well known problem in ORMs. To illustrate the problem let’s say we have this schema:

By implementing __str__ we tell Django that we want the name of the category to be used as the default description of the object. Whenever we print a category object Django will fetch the name of the category.

A simple admin page for our Product model might look like this:

This seems innocent enough but the SQL log reveals the horror:

Django first counts the objects (more on that later), then fetches the actual objects (limiting to the default page size of 100) and then passes the data on to the template for rendering. We used the category name as the description of the Category object so for each product Django has to fetch the category name — this results in 100 additional queries.

To tell Django we want to perform a join instead of fetching the names of the categories one by one,we can use list_select_related

Now the SQL log looks much nicer. Instead of 101 queries we have only 1:

To understand the real impact of this setting consider the following — Django default page size is 100 objects. If you have one related fields you have ~101 queries, if you have two related objects displayed in the list view you have ~201 queries and so on.

Fetching related fields in a join can only work for ForeignKey relations. If you wish to display ManyToMany relations it’s a bit more complicated (and most of the time wrong, but keep reading).

Related Fields

Sometimes it can be useful to quickly navigate between objects. After trying for a while to teach support personnel to filter using URL parameters we finally gave up and created two simple decorators.

admin_link

Create a link to a detail page of a related model:

The decorator will render a link (a) to the related model in both the list view and the detail view. In our case we might want to add a link from each product to its category detail page in both the list view and the detail view:

admin_changelist_link

For more complicated links such as “all the products of a category” we created a decorator that accepts a query string and link to the list view of a related model:

Adding a link from a category to a list of it’s products, the Category admin looks like this:

Be careful with the products argument — it’s very tempting to do something like return ‘see {} products’.format(products.count()) but it will result in additional queries.

readonly_fields

In the detail page Django creates an editable element for each field. Text and numeric fields will be rendered as regular input field. Choice fields and foreign key fields will be rendered as a <select> element. Rendering a select box requires Django to do the following:

  1. Fetch the options — the entire related model + descriptions (remember the N+1 problem?).
  2. Render the option list — one option for each related model instance.

One common example that is often overlooked is foreign key to the User model. When you have 100 users you might not feel the load but what happens when you suddenly have 100K users? The detail page will fetch the entire users table and the option list will make the resulting HTML file huge. We pay twice — first for the full table scan and then for downloading the html file (not to mention the memory required to generate the file).

Having a select box with 100K options is not really usable. The easiest way to prevent Django from rendering a field as a <select> element is to make it a read only field in the admin:

This will render the description of the related model without being able to change it in the admin.

Filters

As mentioned above, we often use the admin interface as a day to day tool for general support. We found that most of the times we use the same filters — only active users, users registered in the last month, successful transactions, etc. Once we realized that, we asked ourselves, why fetch the entire dataset if we are most likely to immediately apply a filter. We started to look for a way to apply a default filter when entering the model list view. Other than the usability perk it also makes paging cheaper.

DefaultFilterMixin

There are many approaches for applying a default filter. Some of them involve fancy custom filters and some inject magic query parameters to the request.

We found this approach to be simple and straightforward:

If the list view was accessed from a different view and no query params were specified we generate a default query and redirect.

Let’s add a default filter to our Product page to filter only products created in the last month:

If we drill down from within the page or if we get to the page with query parameters (using admin_changelist_link for example) the default filter will not be applied.

Profit!

Quick Bits

Some neat tricks we gathered over time:

show_full_result_count

Prevent Django from displaying the total amount of rows in the list view. Setting this option to False saves a count(*) query on the queryset.

defer

When performing a query the entire result set is put into memory for processing. If you have large columns in your model (such as JSON or Text fields) it might be a good idea to defer them until you really need to use them and save some memory. To defer fields override get_queryset (or better yet, create a mixin!).

Change the admin default URL route

This is definitely not the only precaution you should take to protect your admin page but it can make it harder for “curious” users to reach the login page. In your main urls.py override the default admin route:

date_hierarchy

We found this index can improve query when filtering on the date_hierarchy (PostgresSQL 9.4):

Make sure to change table name, index name, the date hierarchy column and the time zone.

Conclusion

Even if you don’t have 100K users and millions of records in the database it is still important to keep the admin tidy. Bad code has this nasty tendency of biting you in the ass when you least expect it.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store