Django’s select_for_update with Examples and Tests

Alexandre Laplante
Nov 19, 2018 · 2 min read

Consider the following Django model.

There is a potential problem with this code. If the publish method is called on an already published article, we will trigger all the side effects of publishing a second time! Let’s try to prevent that.

Is this code safe now? If we only have 1 server, then yes. If we have multiple nodes running the same code all connected to the same database, then no. What happens if two different servers get the command to publish at the same time? Then both servers could get the same article with is_published=False, and both proceed to call the _publish function.

The solution is to use select_for_update to take a database lock.

In the code above we use database locks to force the _publish function only to be called on an article by one server at a time. We’re using a pattern called Double-Checked Lock Synchronization.

How can we test the code above? Django’s own tests use threads and sleep for a fixed number of seconds to get everything in the right state. It is possible to do this without threads or sleeping, if we make some small modifications to our code.

In order to test this, we have to use a database that supports transactions and locks. Django’s select_for_update statement is completely ignored by SQLite. In this example I’m using PostgreSQL.

Locks in PostgreSQL are granted to a database connection. So in order to test this code, we’re going to have to open multiple connections to the same database. We’ll do so by having multiple aliases to the same database in our settings.py file.

Django’s select_for_update will release the lock when the outermost transaction it is inside gets committed. We will use transactions cleverly to arrange a state where locks are still held by one connection while we try to acquire them in another.

First, we have to adjust our code to care about which database connection we’re using.

This code will behave the same as the previous version when no database is provided to publish, but will use the specified database for everything when it is provided.

Now we can write the following tests.

Note that we have to use TransactionTestCase and not TestCase. TestCase wraps every test inside a transaction block which gets rolled back at the end of the test. This is not suitable for our tests, which rely on having control over the outer transaction block. TransactionTestCase doesn’t wrap anything in a transaction, so our code can actually make changes to the test database.

Also note that the only difference between the two tests is the level of indentation in the second publish call. This is the difference between waiting for the first transaction to finish and not.

Alexandre Laplante

Written by

Software Engineer at Google. Quantum Computing, Cryptocurrency. Also regular computing and regular currency.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade