How to debug changes of models in Django

A little bit of context

Usually your Django app is a quite difficult system, where you have models that are linked between each other with a pretty complex logic behind that. And when a user changes something in one object, this can impact a lot of different objects in your database, depending on business logic.

When there is a problem on your back-end and it fails, users of your app, most of the time, don’t stop and try to click on the button again, or even go back and change something so that, they think, they can go further 😄 .

Usually when a developer starts to investigate a bug, they are faced with the current state of the objects in DB. This can be very hard to understand why some actions were triggered and why the current state is like it is (because the user actions was not supposed to change the objects the way they are right now or probably something else happened afterward like another user action or an admin actions for example). Even more complicated is if the current state of the DB is the same as when the bug happened, it may be very helpful to be able to see what was changed and in which order, to understand all the way what the user did.

To solve these problems, I want to speak about one interesting package for Django, that can help you to track all the changes that happened to instances of your models , namely Django Models Logging.

How to use

To setup this package in your project you need to do following:

Installation

Install the package from pypi

pip install django-models-logging

Configuration

Put models_logging into INSTALLED_APPS in your settings.py . It should be added at the very end of INSTALLED_APPS ! Because the package uses signals to track changes, so all apps/models that have to be tracked should be above models_logging to work correctly.

Then you need to add the models you want to track in settings.py , using this:

LOGGING_MODELS = (
    'app.MyModel',    # logging only for this model
    'another_app'     # logging of all models in this app
)

If you do not want to track all models in another_app , you can just put another option

LOGGING_EXCLUDE = (
    'another_app.AnotherModel'     # ignore logging for this model
)

Apply migrations python manage.py migrate , and now if you run your server, you should see this in Django admin

All changes - this is the model that stores all changes (old and new values) of tracked models in your application. And I will tell you later what is Revisions 😉

You can set only specific model fields to be tracked

class MyModel:
    # only changes for "name" field will be tracked
    LOGGING_ONLY_FIELDS  = ("name",)    
    
    name = models.CharField()
    description = models.TextField()

Or exclude some fields

class MyModel:
    # changes of "description" field will be ignored, 
    # all the rest will be stored in database
    LOGGING_IGNORE_FIELDS  = ("description",)

    name = models.CharField()
    description = models.TextField()
    is_available = models.BooleanField()

Now, if you change the MyModel object (even using Django shell), you will see these changes in Django admin in All changes section.

Merge multiple changes into one: Revisions !

Let’s get back to Revisions and what is it exactly. In projects I've worked on, I've often seen objects saved many times during a single request to back-end. This should not happen at all, but sometimes you arrive to the project and you don't have time to fix this problem, but you still need to track changes.

The changes of the tracked model are saved in the database every time when .save() method of the model is called, so it means, if you have a problem of multiple saving during 1 request, it will create a lot of objects of Change model for 1 object. So that, your database can quickly grow to a very large size.

To prevent this “bug” you can add a specific middleware in settings.py

MIDDLEWARE = (
    ...,
    'models_logging.middleware.LoggingStackMiddleware',    
)

It will “merge” all changes of an object during the request. You can also use a specific context manager to have “merging” behavior, if you call some script manually, outside django views, here is an example of manage.py command:

from models_logging.utils import create_merged_changes

class Command(BaseCommand):
    def handle(self, *args, **options):
        with create_merged_changes():
            ...

This context_manager and middleware, they merge changes into Revisions , all changes for all objects that were changed during the request/script will be grouped and linked to the model Revision . Also this middleware allows to put information of the user who changed the object.

Show me the changes!

All changes for tracked models are available in Revisions or All changes sections in Django admin, but it will be more convenient to see it directly on the page of the tracked object. For that the package provides HistoryAdmin .

If you use HistoryAdmin as a parent class for your admin class, you will be able to see changes for the object by clicking History on detailed view.

from models_logging.admin import HistoryAdmin

class MyModelAdmin(HistoryAdmin):
    history_latest_first = False    # latest changes first
    inline_models_history = '__all__'   # __all__ or list of inline models for this ModelAdmin

The inline_models_history attribute allows you to also see changes of all related objects for models that are linked to MyModel with a ForeignKey (only if they are tracked).

There is also one more quite useful feature in this package, you can revert a change or revision (group of changes) at any time. There is a button Revert in Django admin for model Change and Revert . And for this feature there are additional parameters in settings

def can_revert(request, obj):
    return request.user.username == 'myusername'

LOGGING_REVERT_IS_ALLOWED = can_revert    # global parameter
LOGGING_CAN_DELETE_REVISION = can_revert
LOGGING_CAN_DELETE_CHANGES = False
LOGGING_CAN_CHANGE_CHANGES = True

They can be either boolean or a function that expects request and the object itself, so you can control it with permissions.

Something you need to remember

Important points:

1. This package uses Django signals post_init , post_save and post_delete , so if you update an object with .update() method of QuerySet , these signals are not fired, so you will not see tracked changes to the database.
2. This package is not optimal as a backup for your database. If you try to recover thousands of changes this will be very slow.
3. Besides, if you track too many models and apps, your database, probably, will growth too fast, but there is a Django command delete_changes to solve this issue. Most likely you don't need to store all changes for all time, and you can configure some cron task that will run this command with specific parameters to clear the database:

--ctype - ids listed by comma of ContentType which will be deleted

--ctype-exclude - ids listed by comma of ContentType which will be excluded from deletion

--date_lte - The changes started before that date will be removed, format (yyyy.mm.dd)