Blazing fast CI with GitHub Actions, Poetry, Black and Pytest

Setting up GitHub Actions for a modern Django project

Photo by Puk Patrick on Unsplash

Few things are as detrimental to productivity as slow feedback loops—slow tests trickle down to slow product rollouts, slow deployments, slow developer speed; and ultimately a sour developer experience for the team.

Setting up CI is pretty time consuming—it’s usually a trial-and-error shit-show where you spend most of your time watching things passing, passing, passing… and failing on the last line, after 15 minutes and 6000 unit tests.

After a bunch of personal frustration I figured out how to get things running rapidly, and I’d like to share a pragmatic, rapid workflow for a modern Django project — but these steps can be abstracted for JS, Golang, or any codebase.

GitHub Actions (as of this week) supports file system caching, so we can avoid the ever tedious pip install/pipenv install/poetry install step. (Or npm install or whatever your personal flavor of package manager is if you’re not using Python).

Create a new Poetry project

For sake of explanation, let’s start on a blank canvas and create a new Django project from scratch.

Make sure Poetry is installed and create a new Poetry-managed project

cd my_project
poetry init -n

This’ll create a new pyproject.toml file in the current directory. Let’s install Django and Confidential:

poetry add django confidential
Creating virtualenv delme-py3.7 in /Users/dvf/Library/Caches/pypoetry/virtualenvs
Using version ^2.0 for confidential
Using version ^2.2 for django
Updating dependencies
Resolving dependencies... (1.6s)
Writing lock file
Package operations: 13 installs, 0 updates, 0 removals
- Installing six (1.13.0)
  - Installing docutils (0.15.2)
  - Installing jmespath (0.9.4)
  - Installing python-dateutil (2.8.0)
  - Installing urllib3 (1.25.6)
  - Installing botocore (1.13.10)
  - Installing s3transfer (0.2.1)
  - Installing boto3 (1.10.10)
  - Installing click (7.0)
  - Installing pytz (2019.3)
  - Installing sqlparse (0.3.0)
  - Installing confidential (2.0.0)
  - Installing django (2.2.7)

Let’s install some dev dependencies:

poetry add --dev pytest pytest-cov pytest-django pytest-xdist

• pytest is a powerful testing framework for Python.
• pytest-cov tells you how well your tests cover your code by generating a coverage report after a test run.
• pytest-xdist is a plugin for pytest allowing you to easily parallelize tests.
• pytest-django is a plugin for pytest that provides a set of useful tools for testing Django applications and projects.

Now, let’s add Black, a popular Python code formatter:

poetry add --dev black --allow-prereleases

Lastly, let’s create a new Django project (named project):

poetry run django-admin startproject project .

Your project directory should look like this:

.
├── manage.py
├── poetry.lock
├── project
│   ├── __init__.py
│   ├── settings.py
│   ├── urls.py
│   └── wsgi.py
└── pyproject.toml

Now we’re ready to start setting up a fast CI pipeline for development.

Setting up your project for CI

In your project folder, create a new file ci.yml in .github/workflows/.

Let’s start off with an initial workflow that pulls our code and installs Python 3.7 to the running container. Add the following to ci.yml:

name: CI

on: [push]

jobs:

  test:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v1
        with:
          fetch-depth: 1

      - name: Set up Python 3.7
        uses: actions/setup-python@v1
        with:
          python-version: 3.7

After saving, committing and pushing this change to GitHub, head over to the Actions tab on your GitHub repository and check if the workflow is executing.

GitHub Actions executing the project workflow

Managing your variables between environments

Confidential is a utility that can store and decrypt secrets in your app. We’re going to use Confidential to set the values in Django’s settings.py. Let’s create two JSON files to hold our variables. Create a folder called .confidential and inside it, create two files:

default.json (to hold default variables)

{
  "SECRET_KEY": "something",
  "ALLOWED_HOSTS": ["*"],
  "DEBUG": true,
  "DATABASE": {
    "ENGINE": "django.db.backends.sqlite3",
    "NAME": "project.db"
  }
}

ci.json (to hold overrides for our CI environment)

{
  "DATABASE": {
    "NAME": ":memory:"
  },
  "DEBUG": false
}

🤚 Confidential uses AWS Secrets Manager allowing you to save encrypted values to your repository, and decrypt them at run time—since we’re not using encrypted values we don’t have to worry about setting that up, but if you have a production environment then you should also create a prod.json file containing these production keys. Read the Confidential docs for more info.

Let’s set up Confidential in Django. Open up settings.py and add the following to the top of the file:

from confidential import SecretsManager

confidential = SecretsManager(
    secrets_file_default=".confidential/defaults.json",
    secrets_file=os.environ.get("SECRETS_FILE"),
    region_name="us-east-1"
)

Now, override these keys (so that they get their values from Confidential):

SECRET_KEY = confidential["SECRET_KEY"]
DEBUG = confidential["DEBUG"]
ALLOWED_HOSTS = confidential["ALLOWED_HOSTS"]
DATABASES = {
    'default': confidential["DATABASE"]
}

Running the project

Let’s run migrations to seed the database:

poetry run python manage.py migrate
Operations to perform:
  Apply all migrations: admin, auth, contenttypes, sessions
Running migrations:
  Applying contenttypes.0001_initial... OK
  Applying auth.0001_initial... OK
  Applying admin.0001_initial... OK
  Applying admin.0002_logentry_remove_auto_add... OK
  Applying admin.0003_logentry_add_action_flag_choices... OK
  Applying contenttypes.0002_remove_content_type_name... OK
  Applying auth.0002_alter_permission_name_max_length... OK
  Applying auth.0003_alter_user_email_max_length... OK
  Applying auth.0004_alter_user_username_opts... OK
  Applying auth.0005_alter_user_last_login_null... OK
  Applying auth.0006_require_contenttypes_0002... OK
  Applying auth.0007_alter_validators_add_error_messages... OK
  Applying auth.0008_alter_user_username_max_length... OK
  Applying auth.0009_alter_user_last_name_max_length... OK
  Applying auth.0010_alter_group_name_max_length... OK
  Applying auth.0011_update_proxy_permissions... OK
  Applying sessions.0001_initial... OK

Your project directory should now look like so:

.
├── .confidential
│   ├── ci.json
│   └── defaults.json
├── .github
│   └── workflows
│       └── ci.yml
├── manage.py
├── poetry.lock
├── project
│   ├── __init__.py
│   ├── settings.py
│   ├── urls.py
│   └── wsgi.py
├── project.db
└── pyproject.toml

At this point we’re ready to run the project:

poetry run python manage.py runserver
Watching for file changes with StatReloader
Performing system checks...
System check identified no issues (0 silenced).
November 05, 2019 - 21:17:46
Django version 2.2.7, using settings 'project.settings'
Starting development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.

Make sure the server is running by visiting http://127.0.0.1:8000/.

Create some tests

Create a new folder tests and a new file inside it named test_something.py, add some fake tests inside it to verify our pipeline:

# test_something.py
def test_foo():
    assert True


def test_bar():
    assert True

Run the tests:

poetry run pytest
======================= test session starts ========================
Python 3.7.4, pytest-5.2.2, py-1.8.0, pluggy-0.13.0
rootdir: /Users/dvf/code/sample-ci
plugins: django-3.6.0, xdist-1.30.0, forked-1.1.3
collected 2 items
tests/test_something.py ..                                     [100%]
======================== 2 passed in 0.02s =========================

They should pass. Try run them in parallel (2 groups):

poetry run pytest -n 2
======================= test session starts ========================
Python 3.7.4, pytest-5.2.2, py-1.8.0, pluggy-0.13.0
rootdir: /Users/dvf/code/sample-ci
plugins: django-3.6.0, xdist-1.30.0, forked-1.1.3
gw0 [2] / gw1 [2]
..                                                             [100%]
======================== 2 passed in 0.40s =========================

Completing the CI workflow

Here is the entire complete ci.yml workflow, that will run everything (with caching) in GitHub Actions:

name: CI

on: [push]

jobs:

  test:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v1
        with:
          fetch-depth: 1

      - name: Set up Python 3.7
        uses: actions/setup-python@v1
        with:
          python-version: 3.7

      - name: Install Poetry
        uses: dschep/install-poetry-action@v1.2

      - name: Cache Poetry virtualenv
        uses: actions/cache@v1
        id: cache
        with:
          path: ~/.virtualenvs
          key: poetry-${{ hashFiles('**/poetry.lock') }}
          restore-keys: |
            poetry-${{ hashFiles('**/poetry.lock') }}

      - name: Set Poetry config
        run: |
          poetry config settings.virtualenvs.in-project false
          poetry config settings.virtualenvs.path ~/.virtualenvs

      - name: Install Dependencies
        run: poetry install
        if: steps.cache.outputs.cache-hit != 'true'

      - name: Code Quality
        run: poetry run black . --check

      - name: Test with pytest
        env:
          DJANGO_SETTINGS_MODULE: project.settings
          SECRETS_FILE: .confidential/ci.json
        run: poetry run pytest --cov . -n 2

Let’s go through the job steps:

• Install Poetry installs Poetry in your build
• Cache Poetry virtualenv creates a cache in the build pipeline and keys it by the hash of your poetry.lock file, meaning that the cache will be busted if your packages change.
• Set Poetry config tells specifies a cacheable path on the filesystem for the virtualenv to reside in.
• Install Dependencies installs the dependencies using Poetry, but only if there was not a cache hit. This is a huge time saver if you have a large project.
• Code Quality runs Black in check mode against the codebase. If files aren’t formatted correctly the build will fail.
• Test with pytest runs pytest against the test suite by splitting the texts into 2 parallel groups (-n 2). It also tells confidential to use the ci.json file for our variables by setting an environment variable.

Formatting and pushing to GitHub

Firstly, don’t forget to format your code

poetry run black .
reformatted /Users/dvf/code/sample-ci/project/wsgi.py
reformatted /Users/dvf/code/sample-ci/project/urls.py
reformatted /Users/dvf/code/sample-ci/manage.py
reformatted /Users/dvf/code/sample-ci/project/settings.py
All done! ✨ 🍰 ✨
4 files reformatted, 3 files left unchanged.

Now, push this change to GitHub, and navigate to Actions. Let’s make sure that our workflow is executing.

Take note of the time it took to execute our pipeline. In the above run, we see it took 1m 4s . But if you look closer you’ll see that the dependency installation took 33s. Since this step is being cached, on the next run it should take 0s.

Let’s push an innocuous change to the codebase and check the cached time:

That’s more like it, 27s. As the test suite grows we could potentially experiment with increasing the number of test runners from 2 to 4, but it may not make much difference since the current iteration of GitHub Actions has only 2 cores per container.

Hope you found this useful. If you have any other optimization, I’d love to hear from you on twitter @van_flymen!