10 things I learned in 10 years of using Django

About Sebastian

Over the past 10 years working in tech I was lucky enough to work with many extraordinary engineers and learn from them. This post represents the essence of that experience, the top 10 tools & techniques which help my teams and me being successful.

This post is intended as a guideline for beginners as well as a refresher for experts. Applying these principles helped me to rapidly move from ideation to MVP to successful products, to grow the engineers working on a project from one to multiple scrum teams and to pass technical due diligences.

Isolate your environments

Every new application should start by creating an own, isolated environment. For quite a long time, virtualenv & a-likes were the defacto standard for creating an isolated workspace for a Python application and pinning the dependencies for a project in a own requirements.txt.

While this is already a huge improvement, it still leaves room for error as the system where the code is developed is usually (and hopefully) not the exact same one where it later is deployed. Small things such as a missing jpeg library on the production server might crash the application.

That’s where Docker comes in handy. It not just runs your code in the exact same system & environment for development & for production, it also automatically documents your setup.

Separate code & data

There are usually two types of servers: application servers running the code and data servers providing persistence.

Keep them separate & treat your application server as they could die every moment — eventually they will. If you follow this advice, all you need to do is re-deploy the code — and the data is still there.

As I want to create new products and not spend my time with DevOps work, I like to use hosted services such as Amazon Web Services’ RDS for hosting Postgres databases, Heroku for running my code and Cloudinary for hosting media.

These three services are the best thing ever! (Disclaimer: I’m a fanboy)

Love the blue elephant

There are a gazillion of databases out there. Every now and then, something shiny & new comes up. I love the concept of document oriented databases such as MongoDB or CouchDB and have used both in the past. But eventually you want to have something simple & solid — such as Postgres.
Don’t underestimate the power of Postgres — it’s one hell of a database.

Here’s a couple of reasons why I advocate for using Postgres:

• It’s based on SQL: Django’s ORM & many third party extensions rely on SQL. Missing out on these great ecosystem is a huge bummer.
• It’s fast enough. Many NoSQL databases advertise with super fast reads/ writes. If you don’t have very special use case, Postgres will work just fine and is fast enough.
• It’s flexible: Postgres’s JSON data type allows you to store /documents/ and run indexed (!) queries on them without performance losses.

Separate Frontend & Backend

The time where web applications were html pages with some JavaScript is over. Modern frontends are JavaScript heavy, quite complex due to their asynchronous nature and require quite a lot of dependencies to set it up (Node.js, transpiling, webpack, React.js, …).

Following the design principle Separation of Concerns, splitting frontend and backend up is the recommended way.

In modern services, the backend provides data to the frontends via an API in a machine readable format, usually JSON. While GraphQL is the cool new kid on the block, simple REST services get the job often done quite good (and can act as a data provider for GraphQL later on).

The Django community can be really happy to have the awesome library Django Rest Framework which simplifies building REST APIs a lot.
Learn it, use it, love it.

Use celery for async tasks & cron jobs

Communication with external systems always include latency and can be unpredictable. Initially you might not notice a performance decline, but as you add more external services your application might become slow because of that. This can be simple things such as sending emails or triggering notifications.

That’s where the task queue celery comes in handy. Any call that doesn’t need to be synchronous can be queued up and eventually be consumed by a celery worker.

But celery offers more than just that. You can use it for:

• Performing asynchronous tasks such as interaction with an external service (e.g. sending an email).
• Delaying tasks (e.g. sending a welcome mail 10 minutes after the user signed up).
• Executing periodic tasks such as a daily export instead of using cronjobs.

Learn to love Redis

While your application grows, you will sooner or later need caching or a message broker. Redis is an in-memory data store which is excellent at both.

Use it as your queue for celery or just as plain simple caching of views or expensive calculations.

It’s an excellent piece of software and is easy to use. You’ll love it!

Tracking all errors

Once your service runs in production, errors will happen no matter how well you’ve tested. And that’s ok. Just be sure to catch these errors as soon as possible, prioritise them and fix them.

Sentry is an open source error tracking tool (also offered as a SaaS, which I recommend) that helps monitoring & fixing crashes. It’s an absolute indispensable tool for diagnosing errors, in the backend and in the frontends as it gives you detailed context including the stack trace and how often the error occurred.

Always keep in mind, errors can & will happen — just make sure this specific error happens only once. Write a test case for it.

Apply code hygiene from day #1 on

Code quality is essential. If your code smells, your whole service will eventually suffer from it.
That’s why it is your job to insist on highest standards. These standards include:

• Use a linter such as flake8 to analyse your code for potential errors and to enforce a unified coding style.
• Use a coverage tool to calculate your test coverage and to visualise it. That doesn’t tell you how good your tests are, but it’s a great tool to detect untested code.
• Use a Continuous Integration (CI) service for running your linter and your test suite on every push to your remote git repository. Nobody should ever review or deploy code which hasn’t run through these steps first.

I highly recommend to apply those from the very first day. Don’t make sacrifices on the wrong parts.

Document your code

Documenting code is a essential part of any developer. Always treat your code as it would need to hand it over to someone else the next day.

You don’t need to write a full blown handbook (also that’s super nice). Keep it simple and make it a habit.

The following ways of documenting proofed to be very helpful to me in the past:

• 30k feet documentation: This can be as simple as just one chart (we use draw.io for that) describing all internal & external services and how they play together.
• Feature changelog: Describe each new feature with a few sentences and store it in a `changelog` folder in your code base as a text file (we use markdown). Pro tip: Enforce the existence of a changelog file in your CI.
• Inline Docs: Create a habit of starting each new function with a doc string describing what the function does. This makes your code so much easier to read — for other people but also for yourself in a year from now. Descriptive/ verbose function & variable names alone don’t do the trick.

Write code which is easy to test

Django views have a tendency to become huge bloated monsters with lots of business logic inside. As this makes testing quite hard, try to move that code into the models. Write fat models & skinny views.

The smaller a function is, the easier it is to test. So don’t be afraid of splitting your logic up into smaller chunks. This way testing becomes fun.