Open in app

Sign in

Write

Sign in

Base 2 — Linting Django

Piotr Zalewa
Django Unleashed
Published in
10 min readJun 27, 2024

I enjoy the beauty of code, so I’ve chosen Python. The way the code looks is part of the syntax. Writing ugly code in Python is possible, but it’s harder than in other languages. Linting has been an essential part of code development for several years now. Having a team follow one chosen standard increases the readability and mobility of the code.

Linting Django hallucinated by Dall-E3

This is the second article from the series. Please look at Django with Poetry for the first one. We’re starting from the branch part-1 of the djpoe repository. This article will install and configure tools - Ruff, Pylint, Mypy, and pre-commit. In my opinion, these tools are essential to each more immense code repository. We will also install PoeThePoet, which simplifies running management commands.

Ruff

Until recently, I’ve been using a bunch of tools to lint code (flake8, black), keep imports in order (isort) and take care of docstrings (pydocstring). It takes some effort to know them all and to configure them in a way that works well. Then came ruff. “One tool to rule them all.”

Let’s install ruff with Poetry

djpoe ➜ poetry add --group dev ruff
...
  - Installing ruff (0.4.10)
...

Ruff’s main command is check . The documentation mentions the following usage:

ruff check                  # Lint all files in the current directory.
ruff check --fix            # Lint all files in the current directory, and fix any fixable errors.
ruff check --watch          # Lint all files in the current directory, and re-lint on change.
ruff check path/to/code/    # Lint all files in `path/to/code` (and any subdirectories).

We will manually use ruff check. Linting-specific files will be used in pre-commit.

djpoe ➜ poetry run ruff check
All checks passed!

If we create an error in the files, it will be reported. I’ve added an unused import to the manage.py . We can see the error and fix it with these two commands.

djpoe ➜ poetry run ruff check                
djpoe/manage.py:6:8: F401 [*] `pytest` imported but unused
Found 1 error.
[*] 1 fixable with the `--fix` option.
djpoe ➜ poetry run ruff check --fix
Found 1 error (1 fixed, 0 remaining).

The Ruff’s format command is used to format the code. One can configure the VSCode to use it.

djpoe ➜ poetry run ruff format     
1 file reformatted, 9 files left unchanged

The formatted file is urls.py. One line was added after the module’s docstring.

Configuration

We will configure Ruff in our main pyproject.toml file. The full list could look like this:

# pyproject.toml
...

[tool.ruff]
exclude = [
    ".direnv",
    ".git",
    ".git-rewrite",
    ".mypy_cache",
    ".pytest_cache",
    ".ruff_cache",
    ".venv",
    ".vscode",
    "node_modules",
]
line-length = 120
indent-width = 4
target-version = "py312"

[tool.ruff.lint]
ignore = ["ISC001", "D203", "D213"]
select = [
    "ANN", # flake8-annotations
    "ARG", # flake8-arguments
    "B",   # flake8-bugbear
    "C4",  # flake8-comprehensions
    "C90", # mccabe complexity
    "D",   # pydocstyle
    "DJ",  # flake8-django
    "DTZ", # flake8-datetimez
    "E",   # pycodestyle
    "EM",  # flake8-errmsg
    "F",   # flake8
    "G",   # flake8-logging-format
    "I",   # isort
    "ISC", # flake8-implicit-str-concat
    "LOG", # flake8-logging
    "N",   # pep8-naming
    "PIE", # flake8-pie
    "PT",  # flake8-pytest-style
    "PTH", # flake8-use-pathlib
    "RET", # flake8-return
    "RUF", # ruff-specific rules
    "Q",   # flake8-quotes
    "SIM", # flake8-simplify
    "T10", # flake8-print
    "TCH", # flake8-type-checking
    "TRY", # tryceratops
    "UP",  # pyupgrade
]
fixable = ["ALL"]
unfixable = []
# Allow unused variables when underscore-prefixed.
dummy-variable-rgx = "^(_+|(_+[a-zA-Z0-9_]*[a-zA-Z0-9]+?))$"

[tool.ruff.format]
quote-style = "double"
indent-style = "space"
skip-magic-trailing-comma = false
line-ending = "auto"
docstring-code-format = true
docstring-code-line-length = "dynamic"

Most of the settings are self-explanatory. The select list is essential. It defines the tools to be used to lint the code. You can decide to use all of the possible linters. It would be enough to replace that list with ["ALL"]. The ignore list removes conflicting rules.

Without any code change, we’ve got:

djpoe ➜ poetry run ruff format
1 file reformatted, 9 files left unchanged

The formatted file is tests/test_admin_page.py. The code fits in one line as we allow lines to be 120 characters long now.

Running the linters is more worrying…

djpoe ➜ poetry run ruff check 
djpoe/djpoe/__init__.py:1:1: D104 Missing docstring in public package
djpoe/djpoe/asgi.py:1:1: D212 [*] Multi-line docstring summary should start at the first line
djpoe/djpoe/settings.py:1:1: D212 [*] Multi-line docstring summary should start at the first line
djpoe/djpoe/settings.py:90:89: E501 Line too long (91 > 88)
djpoe/djpoe/urls.py:1:1: D212 [*] Multi-line docstring summary should start at the first line
djpoe/djpoe/urls.py:6:1: D413 [*] Missing blank line after last section ("Examples")
djpoe/djpoe/urls.py:6:1: D411 [*] Missing blank line before section ("Examples")
djpoe/djpoe/urls.py:6:1: D407 [*] Missing dashed underline after section ("Examples")
djpoe/djpoe/urls.py:6:1: D406 [*] Section name should end with a newline ("Examples")
djpoe/djpoe/wsgi.py:1:1: D212 [*] Multi-line docstring summary should start at the first line
djpoe/manage.py:8:5: ANN201 Missing return type annotation for public function `main`
djpoe/manage.py:14:15: TRY003 Avoid specifying long messages outside the exception class
djpoe/manage.py:15:13: EM101 Exception must not use a string literal, assign to variable first
tests/__init__.py:1:1: D104 Missing docstring in public package
tests/test_admin_page.py:1:1: D100 Missing docstring in public module
tests/test_admin_page.py:8:1: PT023 [*] Use `@pytest.mark.django_db()` over `@pytest.mark.django_db`
tests/test_admin_page.py:9:5: ANN201 Missing return type annotation for public function `test_render_admin_login`
tests/test_admin_page.py:9:5: D103 Missing docstring in public function
tests/test_admin_page.py:17:1: PT023 [*] Use `@pytest.mark.django_db()` over `@pytest.mark.django_db`
tests/test_admin_page.py:18:5: ANN201 Missing return type annotation for public function `test_admin_redirect_if_not_superuseer`
tests/test_admin_page.py:18:5: D103 Missing docstring in public function
tests/test_admin_page.py:30:1: PT023 [*] Use `@pytest.mark.django_db()` over `@pytest.mark.django_db`
tests/test_admin_page.py:31:5: ANN201 Missing return type annotation for public function `test_admin_success_for_superuser`
tests/test_admin_page.py:31:5: D103 Missing docstring in public function
tests/test_auth_models_user.py:1:1: D100 Missing docstring in public module
tests/test_auth_models_user.py:6:1: PT023 [*] Use `@pytest.mark.django_db()` over `@pytest.mark.django_db`
tests/test_auth_models_user.py:7:5: ANN201 Missing return type annotation for public function `test_user_create`
tests/test_auth_models_user.py:7:5: D103 Missing docstring in public function
tests/test_dummy.py:1:1: D100 Missing docstring in public module
tests/test_dummy.py:1:5: ANN201 Missing return type annotation for public function `test_trivia`
tests/test_dummy.py:1:5: D103 Missing docstring in public function
Found 31 errors.
[*] 12 fixable with the `--fix` option (6 hidden fixes can be enabled with the `--unsafe-fixes` option).

We don’t want some of these errors. We should disable them per file

  • Weather __index__.py has a module docstring or not in a web application doesn’t matter.
  • There is no need to lint docstring or force type definitions in the tests directory.
ignore = ["ISC001", "D203", "D213", "TRY003", "EM101"]

[tool.ruff.lint.per-file-ignores]
"tests/*" = ["D", "ANN"]
"__init__.py" = ["D"]
djpoe ➜ poetry run ruff check
djpoe/djpoe/asgi.py:1:1: D212 [*] Multi-line docstring summary should start at the first line
djpoe/djpoe/settings.py:1:1: D212 [*] Multi-line docstring summary should start at the first line
djpoe/djpoe/urls.py:1:1: D212 [*] Multi-line docstring summary should start at the first line
djpoe/djpoe/urls.py:6:1: D413 [*] Missing blank line after last section ("Examples")
djpoe/djpoe/urls.py:6:1: D411 [*] Missing blank line before section ("Examples")
djpoe/djpoe/urls.py:6:1: D407 [*] Missing dashed underline after section ("Examples")
djpoe/djpoe/urls.py:6:1: D406 [*] Section name should end with a newline ("Examples")
djpoe/djpoe/wsgi.py:1:1: D212 [*] Multi-line docstring summary should start at the first line
djpoe/manage.py:8:5: ANN201 Missing return type annotation for public function `main`
tests/test_admin_page.py:8:1: PT023 [*] Use `@pytest.mark.django_db()` over `@pytest.mark.django_db`
tests/test_admin_page.py:17:1: PT023 [*] Use `@pytest.mark.django_db()` over `@pytest.mark.django_db`
tests/test_admin_page.py:30:1: PT023 [*] Use `@pytest.mark.django_db()` over `@pytest.mark.django_db`
tests/test_auth_models_user.py:6:1: PT023 [*] Use `@pytest.mark.django_db()` over `@pytest.mark.django_db`
Found 13 errors.
[*] 12 fixable with the `--fix` option (1 hidden fix can be enabled with the `--unsafe-fixes` option).

We see 13 errors, and all are fixable.

djpoe ➜ poetry run ruff check --fix
djpoe/manage.py:8:5: ANN201 Missing return type annotation for public function `main`
Found 13 errors (12 fixed, 1 remaining). (1 hidden fix can be enabled with the `--unsafe-fixes` option).
djpoe ➜ poetry run ruff check --fix --unsafe-fixes 
Found 1 error (1 fixed, 0 remaining).
djpoe ➜ poetry run ruff check
All checks passed!

Pylint

Ruff and Pylint are partly overlapping. I like to use pylint, as it’s importing the modules, and we’ll be able to find issues with missing imports.

djpoe ➜ poetry add --group dev pylint pylint-django pylint-per-file-ignores
Using version ^3.2.4 for pylint
Using version ^2.5.5 for pylint-django
Using version ^1.3.2 for pylint-per-file-ignores

Updating dependencies
Resolving dependencies... (0.6s)

Package operations: 9 installs, 0 updates, 0 removals
  - ...
  - Installing pylint (3.2.4)
  - Installing pylint-django (2.5.5)
  - Installing pylint-per-file-ignores (1.3.2)

Writing lock file

We need to configure Pylint.

# pyproject.toml
...

[tool.pylint.'MESSAGES CONTROL']
max-line-length = 120
disable = ["fixme", "too-many-arguments"]
load-plugins = ["pylint_per_file_ignores", "pylint_django"]
django-settings-module = "djpoe.settings"
per-file-ignores = """
    /tests/:missing-function-docstring,assignment-from-no-return,not-context-manager,django-not-configured,imported-auth-user,missing-module-docstring
    /migrations/:invalid-name,missing-class-docstring,wrong-import-order
    models.py:too-many-ancestors
    manage.py:import-outside-toplevel
"""

We’ve disabled checks that allow us to write # FIXME and use methods with several arguments. We’re also ignoring some checks on models.py module, and tests and migrations directory. The import-outside-toplevel is ignored in manage.py as execute_from_command_line is imported inside the try/except block.

djpoe ➜ poetry run pylint tests djpoe/djpoe

-------------------------------------------------------------------
Your code has been rated at 10.00/10

Adding a non-existing import would be reported as:

# tests/test_auth_models_user.py
import nonexistingmodule
djpoe ➜ poetry run pylint tests djpoe
************* Module tests.test_auth_models_user
tests/test_auth_models_user.py:2:0: E0401: Unable to import 'nonexistingmodule' (import-error)
tests/test_auth_models_user.py:2:0: W0611: Unused import nonexistingmodule (unused-import)

-------------------------------------------------------------------
Your code has been rated at 9.21/10 (previous run: 10.00/10, -0.79)

Mypy

Each program I’ve written in the past few years is typed. Typing in programming is crucial as it improves code readability and maintainability and helps catch errors early by ensuring that variables and function signatures adhere to specified types.

djpoe ➜ poetry add --group dev mypy django-stubs

Mypy configuration

Configuring Mypy for Django under Poetry is not so straightforward. Thedjango-stubs module needs to know the location of the settings module. Without change, it would be djpoe.djpoe.settings, unfortunately, that would bite us later during the development of an app. We need to make it django.settings. The simplest way is to modify the PYTHONPATH environment variable to contain the djpoe directory. Poetry has a poetry-dotenv-plugin that reads from the .env file.

djpoe ➜ poetry self add poetry-dotenv-plugin
Using version ^0.2.0 for poetry-dotenv-plugin
...
# .env
PYTHONPATH=$(pwd)/djpoe:$PYTHONPATH

We also need to add Mypy configuration to pyproject.toml

# pyproject.toml

[tool.mypy]
plugins = "mypy_django_plugin.main"
mypy_path = "./djpoe"

[tool.django-stubs]
django_settings_module = "djpoe.settings"
ignore_missing_model_attributes = true

We’re connecting the plugin for Django, specifying the path, and configuring the django-stubs.

Running Mypy

djpoe ➜ poetry run mypy --install-types --check-untyped-defs . 
djpoe/djpoe/settings.py:27: error: Need type annotation for "ALLOWED_HOSTS" (hint: "ALLOWED_HOSTS: list[<type>] = ...")  [var-annotated]
tests/test_admin_page.py:27: error: "_MonkeyPatchedWSGIResponse" has no attribute "url"  [attr-defined]
Found 2 errors in 2 files (checked 10 source files)

Installing missing stub packages:
[...]djpoe/.venv/bin/python -m pip install types-Pygments types-colorama types-setuptools

We directed mypy to install missing types, and it asked if it could do it. You can bypass this question with a -y switch.

It found untyped ALLOWED_HOSTS in settings.py and an error in tests. Let’s fix the settings.py by replacing line nr 27 with:

ALLOWED_HOSTS: list[str] = []

We will ignore the error in the tests/test_admin_page.py file. Also, accidentally, in line 27.

assert response.url == "/admin/login/?next=/admin/"  # type: ignore[attr-defined]
djpoe ➜ poetry run mypy --install-types --check-untyped-defs .
Success: no issues found in 10 source files

PoeThePoet

We have some commands to run now. It’s difficult to remember each one, especially if the project is left for a month or so. Let’s help ourselves with this helpful tool. It needs to be installed like Poetry—outside of the project’s virtual environment.

djpoe ➜ ~/.venv/bin/pip3 install poethepoet
djpoe ➜ ln -fs ~/.venv/bin/poe ~/.local/bin/poe

Let’s add Mypy command:

# pyproject.toml

[tool.poe.tasks.mypy]
cmd = "mypy --install-types --check-untyped-defs ."

And call it

djpoe ➜ poe mypy
Poe => mypy --install-types --check-untyped-defs .
Success: no issues found in 10 source files

I also create shortcuts for standard Django calls to manage.py. It might look like this:

# pyproject.toml

[tool.poe.tasks.test]
help = "Pytest."
cmd = "pytest --showlocals --tb=auto -ra --cov-branch --cov-report=term-missing"

[tool.poe.tasks.mypy]
cmd = "mypy --install-types --check-untyped-defs ."

[tool.poe.tasks.pylint]
cmd = "pylint tests djpoe/djpoe"

[tool.poe.tasks.ruff]
cmd = "ruff check ."

[tool.poe.tasks.check]
sequence = ["mypy", "pylint", "ruff"]

[tool.poe.tasks.format]
cmd = "ruff format ."

[tool.poe.tasks.dev]
help = "Run development server."
cmd = "python ./djpoe/manage.py runserver 127.0.0.1:8001"

[tool.poe.tasks.makemigrations]
help = "Generate new migrations."
cmd = "python ./djpoe/manage.py makemigrations"

[tool.poe.tasks.migrate]
help = "Migrate existing migrations."
cmd = "python ./djpoe/manage.py migrate"

[tool.poe.tasks.manage]
cmd = "python ./djpoe/manage.py"

All checks run in sequence:

djpoe ➜ poe check
Poe => mypy --install-types --check-untyped-defs .
Success: no issues found in 10 source files
Poe => pylint tests djpoe/djpoe

--------------------------------------------------------------------
Your code has been rated at 10.00/10 (previous run: 10.00/10, +0.00)

Poe => ruff check .
All checks passed!

pre-commit

We can use the pre-commit application to ensure nothing is committed without check. It will hook to the git commit command and run linters before the commit happens.

djpoe ➜ brew install pre-commit

The configuration has to be written in the .pre-commit-config.yaml file. It would be great to use one of the existing hooks. Unfortunately, these would run outside of the virtual environment. We will use the language: system style.

fail_fast: true
repos:
  - repo: local
    hooks:
      - id: ruff
        name: ruff
        language: system
        files: "^(?:djpoe|tests)/"
        exclude: "migrations"
        types: [python]
        entry: poetry
        args: ["run", "ruff", "check"]

      - id: pylint
        name: pylint
        language: system
        files: "^(?:djpoe|tests)/"
        exclude: "migrations"
        types: [python]
        entry: poetry
        args: ["run", "pylint", "-sn"]

Installing the hooks

djpoe ➜ pre-commit install 
pre-commit installed at .git/hooks/pre-commit

Committing a yaml file did not trigger any hooks:

djpoe ➜ git commit -am "pre-commit script"
ruff.................................................(no files to check)Skipped
pylint...............................................(no files to check)Skipped
[part-2-linting-and-ci 3025957] pre-commit script
 1 file changed, 21 insertions(+)
 create mode 100644 .pre-commit-config.yaml

An attempt to commit a “broken” python file would fail:

djpoe ➜ git commit -a                     
ruff.....................................................................Failed
- hook id: ruff
- exit code: 1

djpoe/manage.py:7:8: F401 [*] `notexistingmodule` imported but unused
Found 1 error.
[*] 1 fixable with the `--fix` option.

You can find the current code on GitHub. The next article is about running the linters using GitHub Actions.

Errata

Added the pylint-django configuration setting:

django-settings-module = "djpoe.settings"
Django
Python
Pylint
Mypy
Typing

--

--

Piotr Zalewa
Django Unleashed

Written by Piotr Zalewa

457 Followers
Writer for

Creator of JSFiddle, ex-Mozilla dev. Software consultant & mentor. I code and write about programming, mostly Python. Open to diverse technologies.

Help

Status

About

Careers

Press

Blog

Privacy

Terms

Text to speech

Teams