Base 2 — Linting Django
I enjoy the beauty of code, so I’ve chosen Python. The way the code looks is part of the syntax. Writing ugly code in Python is possible, but it’s harder than in other languages. Linting has been an essential part of code development for several years now. Having a team follow one chosen standard increases the readability and mobility of the code.
This is the second article from the series. Please look at Django with Poetry for the first one. We’re starting from the branch part-1
of the djpoe
repository. This article will install and configure tools - Ruff, Pylint, Mypy, and pre-commit. In my opinion, these tools are essential to each more immense code repository. We will also install PoeThePoet, which simplifies running management commands.
Ruff
Until recently, I’ve been using a bunch of tools to lint code (flake8
, black
), keep imports in order (isort
) and take care of docstrings (pydocstring
). It takes some effort to know them all and to configure them in a way that works well. Then came ruff
. “One tool to rule them all.”
Let’s install ruff
with Poetry
djpoe ➜ poetry add --group dev ruff
...
- Installing ruff (0.4.10)
...
Ruff’s main command is check
. The documentation mentions the following usage:
ruff check # Lint all files in the current directory.
ruff check --fix # Lint all files in the current directory, and fix any fixable errors.
ruff check --watch # Lint all files in the current directory, and re-lint on change.
ruff check path/to/code/ # Lint all files in `path/to/code` (and any subdirectories).
We will manually use ruff check
. Linting-specific files will be used in pre-commit
.
djpoe ➜ poetry run ruff check
All checks passed!
If we create an error in the files, it will be reported. I’ve added an unused import to the manage.py
. We can see the error and fix it with these two commands.
djpoe ➜ poetry run ruff check
djpoe/manage.py:6:8: F401 [*] `pytest` imported but unused
Found 1 error.
[*] 1 fixable with the `--fix` option.
djpoe ➜ poetry run ruff check --fix
Found 1 error (1 fixed, 0 remaining).
The Ruff’s format
command is used to format the code. One can configure the VSCode to use it.
djpoe ➜ poetry run ruff format
1 file reformatted, 9 files left unchanged
The formatted file is urls.py
. One line was added after the module’s docstring.
Configuration
We will configure Ruff in our main pyproject.toml
file. The full list could look like this:
# pyproject.toml
...
[tool.ruff]
exclude = [
".direnv",
".git",
".git-rewrite",
".mypy_cache",
".pytest_cache",
".ruff_cache",
".venv",
".vscode",
"node_modules",
]
line-length = 120
indent-width = 4
target-version = "py312"
[tool.ruff.lint]
ignore = ["ISC001", "D203", "D213"]
select = [
"ANN", # flake8-annotations
"ARG", # flake8-arguments
"B", # flake8-bugbear
"C4", # flake8-comprehensions
"C90", # mccabe complexity
"D", # pydocstyle
"DJ", # flake8-django
"DTZ", # flake8-datetimez
"E", # pycodestyle
"EM", # flake8-errmsg
"F", # flake8
"G", # flake8-logging-format
"I", # isort
"ISC", # flake8-implicit-str-concat
"LOG", # flake8-logging
"N", # pep8-naming
"PIE", # flake8-pie
"PT", # flake8-pytest-style
"PTH", # flake8-use-pathlib
"RET", # flake8-return
"RUF", # ruff-specific rules
"Q", # flake8-quotes
"SIM", # flake8-simplify
"T10", # flake8-print
"TCH", # flake8-type-checking
"TRY", # tryceratops
"UP", # pyupgrade
]
fixable = ["ALL"]
unfixable = []
# Allow unused variables when underscore-prefixed.
dummy-variable-rgx = "^(_+|(_+[a-zA-Z0-9_]*[a-zA-Z0-9]+?))$"
[tool.ruff.format]
quote-style = "double"
indent-style = "space"
skip-magic-trailing-comma = false
line-ending = "auto"
docstring-code-format = true
docstring-code-line-length = "dynamic"
Most of the settings are self-explanatory. The select
list is essential. It defines the tools to be used to lint the code. You can decide to use all of the possible linters. It would be enough to replace that list with ["ALL"]
. The ignore
list removes conflicting rules.
Without any code change, we’ve got:
djpoe ➜ poetry run ruff format
1 file reformatted, 9 files left unchanged
The formatted file is tests/test_admin_page.py
. The code fits in one line as we allow lines to be 120 characters long now.
Running the linters is more worrying…
djpoe ➜ poetry run ruff check
djpoe/djpoe/__init__.py:1:1: D104 Missing docstring in public package
djpoe/djpoe/asgi.py:1:1: D212 [*] Multi-line docstring summary should start at the first line
djpoe/djpoe/settings.py:1:1: D212 [*] Multi-line docstring summary should start at the first line
djpoe/djpoe/settings.py:90:89: E501 Line too long (91 > 88)
djpoe/djpoe/urls.py:1:1: D212 [*] Multi-line docstring summary should start at the first line
djpoe/djpoe/urls.py:6:1: D413 [*] Missing blank line after last section ("Examples")
djpoe/djpoe/urls.py:6:1: D411 [*] Missing blank line before section ("Examples")
djpoe/djpoe/urls.py:6:1: D407 [*] Missing dashed underline after section ("Examples")
djpoe/djpoe/urls.py:6:1: D406 [*] Section name should end with a newline ("Examples")
djpoe/djpoe/wsgi.py:1:1: D212 [*] Multi-line docstring summary should start at the first line
djpoe/manage.py:8:5: ANN201 Missing return type annotation for public function `main`
djpoe/manage.py:14:15: TRY003 Avoid specifying long messages outside the exception class
djpoe/manage.py:15:13: EM101 Exception must not use a string literal, assign to variable first
tests/__init__.py:1:1: D104 Missing docstring in public package
tests/test_admin_page.py:1:1: D100 Missing docstring in public module
tests/test_admin_page.py:8:1: PT023 [*] Use `@pytest.mark.django_db()` over `@pytest.mark.django_db`
tests/test_admin_page.py:9:5: ANN201 Missing return type annotation for public function `test_render_admin_login`
tests/test_admin_page.py:9:5: D103 Missing docstring in public function
tests/test_admin_page.py:17:1: PT023 [*] Use `@pytest.mark.django_db()` over `@pytest.mark.django_db`
tests/test_admin_page.py:18:5: ANN201 Missing return type annotation for public function `test_admin_redirect_if_not_superuseer`
tests/test_admin_page.py:18:5: D103 Missing docstring in public function
tests/test_admin_page.py:30:1: PT023 [*] Use `@pytest.mark.django_db()` over `@pytest.mark.django_db`
tests/test_admin_page.py:31:5: ANN201 Missing return type annotation for public function `test_admin_success_for_superuser`
tests/test_admin_page.py:31:5: D103 Missing docstring in public function
tests/test_auth_models_user.py:1:1: D100 Missing docstring in public module
tests/test_auth_models_user.py:6:1: PT023 [*] Use `@pytest.mark.django_db()` over `@pytest.mark.django_db`
tests/test_auth_models_user.py:7:5: ANN201 Missing return type annotation for public function `test_user_create`
tests/test_auth_models_user.py:7:5: D103 Missing docstring in public function
tests/test_dummy.py:1:1: D100 Missing docstring in public module
tests/test_dummy.py:1:5: ANN201 Missing return type annotation for public function `test_trivia`
tests/test_dummy.py:1:5: D103 Missing docstring in public function
Found 31 errors.
[*] 12 fixable with the `--fix` option (6 hidden fixes can be enabled with the `--unsafe-fixes` option).
We don’t want some of these errors. We should disable them per file
- Weather
__index__.py
has a module docstring or not in a web application doesn’t matter. - There is no need to lint docstring or force type definitions in the
tests
directory.
ignore = ["ISC001", "D203", "D213", "TRY003", "EM101"]
[tool.ruff.lint.per-file-ignores]
"tests/*" = ["D", "ANN"]
"__init__.py" = ["D"]
djpoe ➜ poetry run ruff check
djpoe/djpoe/asgi.py:1:1: D212 [*] Multi-line docstring summary should start at the first line
djpoe/djpoe/settings.py:1:1: D212 [*] Multi-line docstring summary should start at the first line
djpoe/djpoe/urls.py:1:1: D212 [*] Multi-line docstring summary should start at the first line
djpoe/djpoe/urls.py:6:1: D413 [*] Missing blank line after last section ("Examples")
djpoe/djpoe/urls.py:6:1: D411 [*] Missing blank line before section ("Examples")
djpoe/djpoe/urls.py:6:1: D407 [*] Missing dashed underline after section ("Examples")
djpoe/djpoe/urls.py:6:1: D406 [*] Section name should end with a newline ("Examples")
djpoe/djpoe/wsgi.py:1:1: D212 [*] Multi-line docstring summary should start at the first line
djpoe/manage.py:8:5: ANN201 Missing return type annotation for public function `main`
tests/test_admin_page.py:8:1: PT023 [*] Use `@pytest.mark.django_db()` over `@pytest.mark.django_db`
tests/test_admin_page.py:17:1: PT023 [*] Use `@pytest.mark.django_db()` over `@pytest.mark.django_db`
tests/test_admin_page.py:30:1: PT023 [*] Use `@pytest.mark.django_db()` over `@pytest.mark.django_db`
tests/test_auth_models_user.py:6:1: PT023 [*] Use `@pytest.mark.django_db()` over `@pytest.mark.django_db`
Found 13 errors.
[*] 12 fixable with the `--fix` option (1 hidden fix can be enabled with the `--unsafe-fixes` option).
We see 13
errors, and all are fixable.
djpoe ➜ poetry run ruff check --fix
djpoe/manage.py:8:5: ANN201 Missing return type annotation for public function `main`
Found 13 errors (12 fixed, 1 remaining). (1 hidden fix can be enabled with the `--unsafe-fixes` option).
djpoe ➜ poetry run ruff check --fix --unsafe-fixes
Found 1 error (1 fixed, 0 remaining).
djpoe ➜ poetry run ruff check
All checks passed!
Pylint
Ruff and Pylint are partly overlapping. I like to use pylint
, as it’s importing the modules, and we’ll be able to find issues with missing imports.
djpoe ➜ poetry add --group dev pylint pylint-django pylint-per-file-ignores
Using version ^3.2.4 for pylint
Using version ^2.5.5 for pylint-django
Using version ^1.3.2 for pylint-per-file-ignores
Updating dependencies
Resolving dependencies... (0.6s)
Package operations: 9 installs, 0 updates, 0 removals
- ...
- Installing pylint (3.2.4)
- Installing pylint-django (2.5.5)
- Installing pylint-per-file-ignores (1.3.2)
Writing lock file
We need to configure Pylint.
# pyproject.toml
...
[tool.pylint.'MESSAGES CONTROL']
max-line-length = 120
disable = ["fixme", "too-many-arguments"]
load-plugins = ["pylint_per_file_ignores", "pylint_django"]
django-settings-module = "djpoe.settings"
per-file-ignores = """
/tests/:missing-function-docstring,assignment-from-no-return,not-context-manager,django-not-configured,imported-auth-user,missing-module-docstring
/migrations/:invalid-name,missing-class-docstring,wrong-import-order
models.py:too-many-ancestors
manage.py:import-outside-toplevel
"""
We’ve disabled checks that allow us to write # FIXME
and use methods with several arguments. We’re also ignoring some checks on models.py
module, and tests
and migrations
directory. The import-outside-toplevel
is ignored in manage.py
as execute_from_command_line
is imported inside the try/except
block.
djpoe ➜ poetry run pylint tests djpoe/djpoe
-------------------------------------------------------------------
Your code has been rated at 10.00/10
Adding a non-existing import would be reported as:
# tests/test_auth_models_user.py
import nonexistingmodule
djpoe ➜ poetry run pylint tests djpoe
************* Module tests.test_auth_models_user
tests/test_auth_models_user.py:2:0: E0401: Unable to import 'nonexistingmodule' (import-error)
tests/test_auth_models_user.py:2:0: W0611: Unused import nonexistingmodule (unused-import)
-------------------------------------------------------------------
Your code has been rated at 9.21/10 (previous run: 10.00/10, -0.79)
Mypy
Each program I’ve written in the past few years is typed. Typing in programming is crucial as it improves code readability and maintainability and helps catch errors early by ensuring that variables and function signatures adhere to specified types.
djpoe ➜ poetry add --group dev mypy django-stubs
Mypy configuration
Configuring Mypy for Django under Poetry is not so straightforward. Thedjango-stubs
module needs to know the location of the settings
module. Without change, it would be djpoe.djpoe.settings
, unfortunately, that would bite us later during the development of an app. We need to make it django.settings
. The simplest way is to modify the PYTHONPATH
environment variable to contain the djpoe
directory. Poetry has a poetry-dotenv-plugin
that reads from the .env
file.
djpoe ➜ poetry self add poetry-dotenv-plugin
Using version ^0.2.0 for poetry-dotenv-plugin
...
# .env
PYTHONPATH=$(pwd)/djpoe:$PYTHONPATH
We also need to add Mypy configuration to pyproject.toml
# pyproject.toml
[tool.mypy]
plugins = "mypy_django_plugin.main"
mypy_path = "./djpoe"
[tool.django-stubs]
django_settings_module = "djpoe.settings"
ignore_missing_model_attributes = true
We’re connecting the plugin for Django, specifying the path, and configuring the django-stubs
.
Running Mypy
djpoe ➜ poetry run mypy --install-types --check-untyped-defs .
djpoe/djpoe/settings.py:27: error: Need type annotation for "ALLOWED_HOSTS" (hint: "ALLOWED_HOSTS: list[<type>] = ...") [var-annotated]
tests/test_admin_page.py:27: error: "_MonkeyPatchedWSGIResponse" has no attribute "url" [attr-defined]
Found 2 errors in 2 files (checked 10 source files)
Installing missing stub packages:
[...]djpoe/.venv/bin/python -m pip install types-Pygments types-colorama types-setuptools
We directed mypy
to install missing types, and it asked if it could do it. You can bypass this question with a -y
switch.
It found untyped ALLOWED_HOSTS
in settings.py
and an error in tests. Let’s fix the settings.py
by replacing line nr 27
with:
ALLOWED_HOSTS: list[str] = []
We will ignore the error in the tests/test_admin_page.py
file. Also, accidentally, in line 27
.
assert response.url == "/admin/login/?next=/admin/" # type: ignore[attr-defined]
djpoe ➜ poetry run mypy --install-types --check-untyped-defs .
Success: no issues found in 10 source files
PoeThePoet
We have some commands to run now. It’s difficult to remember each one, especially if the project is left for a month or so. Let’s help ourselves with this helpful tool. It needs to be installed like Poetry—outside of the project’s virtual environment.
djpoe ➜ ~/.venv/bin/pip3 install poethepoet
djpoe ➜ ln -fs ~/.venv/bin/poe ~/.local/bin/poe
Let’s add Mypy command:
# pyproject.toml
[tool.poe.tasks.mypy]
cmd = "mypy --install-types --check-untyped-defs ."
And call it
djpoe ➜ poe mypy
Poe => mypy --install-types --check-untyped-defs .
Success: no issues found in 10 source files
I also create shortcuts for standard Django calls to manage.py
. It might look like this:
# pyproject.toml
[tool.poe.tasks.test]
help = "Pytest."
cmd = "pytest --showlocals --tb=auto -ra --cov-branch --cov-report=term-missing"
[tool.poe.tasks.mypy]
cmd = "mypy --install-types --check-untyped-defs ."
[tool.poe.tasks.pylint]
cmd = "pylint tests djpoe/djpoe"
[tool.poe.tasks.ruff]
cmd = "ruff check ."
[tool.poe.tasks.check]
sequence = ["mypy", "pylint", "ruff"]
[tool.poe.tasks.format]
cmd = "ruff format ."
[tool.poe.tasks.dev]
help = "Run development server."
cmd = "python ./djpoe/manage.py runserver 127.0.0.1:8001"
[tool.poe.tasks.makemigrations]
help = "Generate new migrations."
cmd = "python ./djpoe/manage.py makemigrations"
[tool.poe.tasks.migrate]
help = "Migrate existing migrations."
cmd = "python ./djpoe/manage.py migrate"
[tool.poe.tasks.manage]
cmd = "python ./djpoe/manage.py"
All checks run in sequence:
djpoe ➜ poe check
Poe => mypy --install-types --check-untyped-defs .
Success: no issues found in 10 source files
Poe => pylint tests djpoe/djpoe
--------------------------------------------------------------------
Your code has been rated at 10.00/10 (previous run: 10.00/10, +0.00)
Poe => ruff check .
All checks passed!
pre-commit
We can use the pre-commit application to ensure nothing is committed without check. It will hook to the git commit
command and run linters before the commit happens.
djpoe ➜ brew install pre-commit
The configuration has to be written in the .pre-commit-config.yaml
file. It would be great to use one of the existing hooks. Unfortunately, these would run outside of the virtual environment. We will use the language: system
style.
fail_fast: true
repos:
- repo: local
hooks:
- id: ruff
name: ruff
language: system
files: "^(?:djpoe|tests)/"
exclude: "migrations"
types: [python]
entry: poetry
args: ["run", "ruff", "check"]
- id: pylint
name: pylint
language: system
files: "^(?:djpoe|tests)/"
exclude: "migrations"
types: [python]
entry: poetry
args: ["run", "pylint", "-sn"]
Installing the hooks
djpoe ➜ pre-commit install
pre-commit installed at .git/hooks/pre-commit
Committing a yaml
file did not trigger any hooks:
djpoe ➜ git commit -am "pre-commit script"
ruff.................................................(no files to check)Skipped
pylint...............................................(no files to check)Skipped
[part-2-linting-and-ci 3025957] pre-commit script
1 file changed, 21 insertions(+)
create mode 100644 .pre-commit-config.yaml
An attempt to commit a “broken” python
file would fail:
djpoe ➜ git commit -a
ruff.....................................................................Failed
- hook id: ruff
- exit code: 1
djpoe/manage.py:7:8: F401 [*] `notexistingmodule` imported but unused
Found 1 error.
[*] 1 fixable with the `--fix` option.
You can find the current code on GitHub. The next article is about running the linters using GitHub Actions.
Errata
Added the pylint-django
configuration setting:
django-settings-module = "djpoe.settings"