Python-Scripts

poetry run start

Here at Octopus Wealth I work on a number of projects that have a Typescript frontend and Python backend. Switching from one to another requires me to remember how the tooling changes, for example in the Typescript code-base I can run yarn run test (or npm run test) whereas in Python I’d run pytest tests/. Happily I believe I’ve found a way for Python to follow Typescript, poetry run test which this article explains.

The scripts in question. Image created using http://instaco.de.

package.json scripts/yarn-scripts

The package.json file present in Javascript and Typescript code-bases has an optional scripts section that maps a single name to a custom command that crucially is run in the installed environment. This means I can execute yarn run lint and have eslint src/**/*.js src/**/*.jsx run (using the eslint installed with the project). I typically then have the following mappings in my package.json file,

"scripts": {
    "start": "node scripts/start.js",
    "format": "prettier --list-different \"src/**/*.{js,jsx}\"",
    "reformat": "prettier --write \"src/**/*.{js,jsx}\"",
    "lint": "eslint src/**/*.{ts,tsx}",
    "test": "jest --env=jsdom",
  },

Aside on the scripts

The scripts are meant to provide individual CI commands to check the formatting, lint and test the code-base, and to provide useful development-only helpers to start and reformat the code-base. The naming of format and reformat could be improved.

pyproject.toml scripts/poetry scripts

Poetry is a Python packaging and dependency management tool. Like yarn it manages dependencies conceptually and via a lockfile with the package.json equivalent called pyproject.toml and the yarn.lock equivalent called poetry.lock. This alone means that I can simply run yarn install or poetry install to install and setup the dependencies for either code-base.

Poetry also understands a scripts section in the pyproject.toml file, sadly though poetry does not yet allow for the command to be specified, rather the script must map to a Python module and function in the format module:function. To get around this I’ve added this to the pyproject.toml file,

[tool.poetry.scripts]
format = "scripts:format"
reformat = "scripts:reformat"
lint = "scripts:lint"
start = "scripts:start"
test = "scripts:test"

then I’ve created a file called scripts.py in the code-base root with the following code,

# This is a temporary workaround till Poetry supports scripts, see
# https://github.com/sdispater/poetry/issues/241.
from subprocess import check_call


def format() -> None:
    check_call(
        ["black", "--check", "--diff", "src/", "tests/"],
    )


def reformat() -> None:
    check_call(["black", "src/", "tests/"])


def lint() -> None:
    check_call(["flake8", "src/", "tests/"])
    check_call(["mypy", "src/backend/", "tests/"])


def start() -> None:
    check_call(["python", "src/backend/run.py"])


def test() -> None:
    check_call(["pytest", "tests/"])

The key part is that check_call will raise an exception if the call returns a non-zero code, which in turn ensures that the poetry call returns a non-zero code. This means that standard CI scripts can understand success and failure.

Conclusion

With this setup my colleagues and I only need to remember that any tool I use in the Javascript/Typescript code-base as yarn run abc is available in the Python code-base as poetry run abc. In detail this is given in the table below,

Aim                    | Javascript          | Python           
--------------------------------------------------------------------
Install dependencies   | yarn install        | poetry install
Lint the code          | yarn run lint       | poetry run lint
Test the code          | yarn run test       | poetry run test
Check formatting       | yarn run format     | poetry run format
Format the code        | yarn run reformat   | poetry run reformat

With thanks to this suggestion that inspired this.