Base 3 — Lint and test with GitHub Actions

In the last two articles (Django with Poetry and Linting Django) we’ve created a plain Django project and added linters. We’ve made them run automatically on each commit so as not to pollute the repository with simple errors. Indeed, it’s still possible to publish without running pre-commit hooks. One can disable it or bypass the check with the
--no-verify switch. To be sure all merged code is linted correctly, we need to add the checks on the repository side.

Microsoft Designer hallucinating the article’s title

For this, we will utilize GitHub Actions. This feature allows us to run a code on a virtual machine. Let’s create the .github/workflows directory and write our first action.

djpoe ➜ mkdir -p .github/workflows

We will define our workflow by creating the check.yml file with the following directives:

# .github/workflows/check.yml

name: Check project

on:
  - push
  - pull_request
jobs:
  check:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: 3.12

      - name: Install Poetry
        uses: snok/install-poetry@v1
        with:
          virtualenvs-in-project: true
          virtualenvs-path: .venv

Let’s go through it step by step. I will not discuss the name as it’s self-explanatory.

on:

It’s a list defining the actions to which this workflow will respond. We use a simple push and pull_request actions, but these can be more sophisticated. This example workflow will respond when someone pushes a tag prefaced with sandbox- :

on:
  push:
    tags:
      - sandbox-*

jobs:

Defines an object containing one or more jobs to be run. Each job runs in a separate environment. One can communicate between jobs by using artifacts or variables.

We’ve created one job called check. Each job requires a definition of the system to boot for it.

runs-on: ubuntu-latest

Then, we define the steps. For now, we only use actions that have already been prepared.

- uses: actions/checkout@v4

Pulls the code from the repository. Each pre-build action has its documentation.

- uses: actions/setup-python@v5
  with:
    python-version: 3.12

Installs python in version 3.12. Here, we pass the parameters to the action using the with object. You can check other parameters in the README file.

- name: Install Poetry
  uses: snok/install-poetry@v1
  with:
    version: 1.8.3
    virtualenvs-create: true
    virtualenvs-in-project: true
    installer-parallel: true

Installs Poetry. We will force it to create the virtual environment in the project’s directory.

Let’s commit, push, and see GitHub Actions:

djpoe ➜ git commit -m "GHA install poetry" --set-upstream origin part-3-check-with-github-actions
djpoe ➜ git push

GitHub Actions page displaying last actions

The check.yml workflow pagee

The check job with expanded Intall Poetry step

Ruff

We now have the environment ready to do some real action. Let’s install dependencies and check the code with ruff.

- name: Install dependencies
  run: poetry install --no-interaction --no-root

- name: Ruff
  run: poetry run ruff check .

Here, we’re using run to define commands executed on the virtual server. We can run multiline commands using the pipe character |

- name: Ruff
  run: |
    source .venv/bin/activate
    ruff check .

Successful run of Ruff

Let’s make a failing test:

# manage.py

import doesnotexist

djpoe ➜ git add djpoe manage.py
djpoe ➜ git commit -m "GHA Test failing ruff" --no-verify
djpoe ➜ git push
djpoe ➜ git reset --hard HEAD~1

We’ve committed a file with a broken import, pushed it to the repository, and deleted the commit to reclaim the clean repository.

Workflow runs with a failed ruff test

Failed Ruff job

Mypy

New step definition:

- name: Mypy
  run: poetry run mypy --install-types --check-untyped-defs --non-interactive .

djpoe ➜ git commit -m "GHA Mypy configuration"
djpoe ➜ git push --force

We need to push with--force as we’ve deleted a pushed commit.

Pytest

We want to run pytest as well. This might be omitted in pre-commit as in some projects it takes a long time.

- name: Pytest
  run: poetry run pytest --showlocals --tb=auto -ra --cov-branch --cov-report=term-missing

It runs perfectly as well.

We see the coverage at 72%. It’s way too low, but looking at files that aren’t tested, it’s not as bad. We can disable them later, for now let’s define our minimal test coverage using--cov-fail-under=95.

- name: Pytest
  run: poetry run pytest --showlocals --tb=auto -ra --cov-branch --cov-report=term-missing --cov-fail-under=95

FAIL Required test coverage of 95% not reached. Total coverage: 72.41%

Let’s omit not tested files in pyproject.toml file.

[tool.coverage.run]
omit = [
    # Omit asgi and wsgi files.
    "djpoe/djpoe/asgi.py",
    "djpoe/djpoe/wsgi.py",
]

Required test coverage of 95% reached. Total coverage: 100.00%

Caching

Looking at the check job, we see some steps took more time than the others.

Overall it took 39s. It’s not much, but we have no actual code yet. The significant time consuming actions are Install Poetry, Mypy, and Install dependencies. Install Poetry downloads a small installation script from GitHub so that caching wouldn’t help.

We don’t have many dependencies as we haven’t started coding yet. Installing dependencies already takes 4s. We can easily cache them following the action’s documentation.

- uses: actions/setup-python@v5
  id: setup-python
  with:
    python-version: 3.12

- name: Install Poetry
  # [...]
  # no change here, removed in article for readability

- name: Load cached venv
  id: cached-poetry-dependencies
  uses: actions/cache@v4
  with:
    path: .venv
    key: venv-${{ runner.os }}-${{ steps.setup-python.outputs.python-version }}-${{ hashFiles('**/poetry.lock') }

- name: Install dependencies
  if: steps.cached-poetry-dependencies.outputs.cache-hit != 'true'
  run: poetry install --no-interaction --no-root

Here, we need to change two steps and add one.

We’ve added id to the setup Python step. It’s needed as we use the output from it later on.

Load cached venv is using the actions/cache action. It caches the .venv path. It will update the cache each time the key changes. The key is built from runner.os that depends on the check job setting, the python version received from the setup-python step’s output, and a hash created from the poetry.lock file.

We’ve added an if condition to the Install dependencies step. It will be run only if the cache-hit output of the cached-poetry-dependencies is not "true".

After running it, we see the Cache not found message, which is expected, but also, the Mypy step is failing.

After some digging, we can add the workaround to the mypy command and try again.

- name: Mypy
  run: poetry run mypy --install-types --check-untyped-defs --non-interactive  --cache-dir=.mypy_cache/ .

Unfortunately, the Cache not found message appears again. We can look into the Install poetry step, and we see a not wanted message:

Run snok/install-poetry@v1
  with:
    version: 1.8.3
    virtualenvs-create: true
    virtualenvs-in-project: true
    installer-parallel: true
    virtualenvs-path: {cache-dir}/virtualenvs

The poetry install command was not using the .venv directory. It might be an error in the action’s documentation. Let’s create an issue, the fix is quite simple, so I’ve attached the pull request as well. We need to either change the cached directory or the virtualenvs-path. Let’s do the latter.

- name: Install Poetry
  uses: snok/install-poetry@v1
  with:
    version: 1.8.3
    virtualenvs-create: true
    virtualenvs-in-project: true
    installer-parallel: true
    virtualenvs-path: .venv

The cache is now restored, and we skimmed a precious 1s as Load cache venv took 3s and Install dependencies was omitted. This will change in the future after some dependencies will be installed. We can also see the key containing Linux, 3.12.4, and a hash string.

To fix the Mypy step, we’ve set the cache directory, and it no longer installs the types every time. Overall, we shaved about 5s. It changes from run to run.

You can find the code in the public repository.

Follow me for more parts. I want to create the blueprint for a deployable Django with CMS by Wagtail.

Update 1

The pull request to snok/install-poetry got merged \o/

Update 2

The next part got published: #4 Deploy on DigitalOcean.