Django: Things I wish I knew before using it (Part I)

Being simple, with great documentation, supportive community and plenty of third-party packages, Django remains any start-up’s go-to web framework. Without a doubt, it is one of the most popular python web frameworks out there. But working with Django is not always as smooth as it seems, especially as the codebase keeps growing.

This will be a series of posts on the lessons we learnt by using Django in production for more than 7 years, with 350k lines of code. In this one, we will delve into the topic of modularisation.

Django’s built-in modularisation and its limitations.

The level of encapsulation provided by Django’s built-in modularisation technique is very basic. It has two abilities: to create different apps and create different modules within the apps (models.py, views.py and urls.py). Further on, you have to define your own conventions as Django has no hard guidelines. This means one app can import any module from another app, like models.py. This leads to tight coupling between apps and exposes private implementation of one app to others. This would soon lead us to a bowlful of spaghetti codes if not careful.

Let’s dive-in: An example.

Let’s try building a premium streaming service application for HealthifyMe Users.

django-admin startproject healthify_tv
cd healthify_tv

The django-admin creates a folder/module structure as shown below:

.
├── healthify_tv
│   ├── __init__.py
│   ├── settings.py
│   ├── urls.py
│   └── wsgi.py
└── manage.py

For our use case, we need a payment app, to handle in-app payments and a recommender app to take care of the content shown to the users.

python3 manage.py startapp recommender
python3 manage.py startapp payment

Django by-default creates modularized boilerplate modules as shown below.

.
├── healthify_tv
│   ├── __init__.py
│   ├── settings.py
│   ├── urls.py
│   └── wsgi.py
├── manage.py
├── payment
│   ├── __init__.py
│   ├── admin.py
│   ├── apps.py
│   ├── migrations
│   │   └── __init__.py
│   ├── models.py
│   ├── tests.py
│   └── views.py
└── recommender
    ├── __init__.py
    ├── admin.py
    ├── apps.py
    ├── migrations
    │   └── __init__.py
    ├── models.py
    ├── tests.py
    └── views.py

Till now all looks neatly organised. Consider the model definition of payment.

Let’s consider a use case where the recommender app has to show different content for free and premium users. The simplest way to do this would be:

This looks good. But there is an underlying issue here. We have unknowingly accessed the private implementation of the payment app, as python DOES NOT RESTRICT us from importing one app’s module into others. Let’s see how this pans out in the future.

The product now decides to let customers pause their plan. Are you thinking of a state now for membership?! Yes, you are on track. Let’s go ahead and add a state field to our Membership model.

So at this point, the definition of the premium member has changed. A user who has paused the subscription is not a premium member even if the end date is after the current date. Therefore we now have to change recommender/views.py’s get_content_for_user method because payment changed the definition of active membership. This is tight coupling. We used an example where only one method needed fixing. Now imagine dozens of apps, with 100 of methods, tightly coupled with each other. Internal changes in one app would become a ripple in the calm waters and affect all other apps!

Lesson learnt:

Models of an app is a private implementation of the app. Directly importing it and accessing breaks the encapsulation of the app.

Here’s a fix for the above problem. Let’s explicitly define the public interface of each app in a single place. Say, in payment/public.py.

We can now use this interface to get any information instead of using models of a different app. This means that despite changes to the internal implementation, all callers will still get the same object in return. The bottom-line idea remains: even when the implementation changes internally, the contract remains unchanged.

We now have a lean public interface to abstract out the functionality of the app through public.py to other apps.

However, we need to remain very cautious while putting any method into public.py, as each one is a promise our app is making to other apps, and the fewer the promises made, the better. In order to keep the public method lean and not loaded with business logic, all helper methods of implementing specific functionality need to be kept in a separate file. App scoped private methods can be placed in a module name starting with underscore e.g. _helper.py or _utils.py or generically _<name>.py. By convention, methods defined in this can only be accessed by other modules within the app.

Finally, a complete Django-app should look something like below.

app_name
  ├── __init__.py
  ├── _constants.py
  ├── _helpers.py
  ├── _utils.py
  ├── admin.py
  ├── apps.py
  ├── migrations
  │   └── __init__.py
  ├── models.py
  ├── public.py
  ├── receivers.py
  ├── signals.py
  ├── tasks.py
  ├── tests.py
  └── views.py

Using public and private not only avoid huge refactoring of code when business changes, but it also solves:

1. DRY. Developers end up writing methods, implementing the exact same functionality in multiple apps, making business logic changes extremely difficult and error-prone.
2. SRP: Each module’s functionality is well established and structured, also each method in the module is responsible for a single part of the functionality. This helps encapsulate the behaviour within the methods and apps respectively.

Conclusion:

Django is, without a doubt, a great framework. But it lacks in-built support for encapsulation. Just by following certain simple conventions, we can improve the maintainability of the code base drastically, thus in the long term avoiding issues like tight coupling between apps and interface bloating.

Always remember…

1. Treat Django apps as services. Their functionality and behaviour need to be encapsulated. Expose functionality of apps using thin and easily understandable interfaces.
2. Abstraction like public/private helps in avoiding repetition of codes and also from exposing private implementations of the app to other apps.