Keeping python code clean with pre-commit hooks: black, flake8 and isort

Automate code formatting tasks with the pre-commit framework

Rohit Gupta
Nov 20, 2019 · 5 min read
Python code snippet.
Python code snippet.
Python code snippet photo by Chris Ried on Unsplash

At a fast-paced startup, where many commits are pushed frequently across various projects and multiple merge requests are created daily, reviewing them and maintaining code quality standards like code compliance with PEP8 all at the same time is a huge challenge.

Making peeps correct their code format or telling them to make changes to follow the pep8 standard is irritating for both.

Sometimes comments on merge requests may sound very nitpicky, like “can you please put two blank lines between two function definition?”, “can you please remove extra spaces?”, “can you please fix imports in place on top?”.

We can delegate this task to tools for maintaining code formatting and styling. So, instead of humans correcting the linting mistakes, we let the tool report devs for their errors. And the “tool” here is the pre-commit hooks. Also, it let us fail fast in the software development cycle.

Workflow with pre-commit hooks

When I commit staged Python files, before committing pre-commit hooks are executed - isort sorts imports then black formats the code and at last flake8 checks code compliance with PEP8. If every check passes, the commit is made else, code is automatically formatted and sent back for review. I then just review the changes made (and perform any other changes if required) and commit again.
So, less time is spent on code formatting and more on code logic & implementation.

In this blog post, I’ll discuss how I set up the pre-commit hooks’ pipeline using the pre-commit framework.

First I’ll describe in brief the tool/utility used in the pipeline:

  • isort — sorts all your imports!

When all hooks combined in the pipeline the workflow look like this:

pre-commit hooks pipeline for checking python files

First, we’ll set up the pre-commit framework then add isort, black, and flake8 in the pipeline.

Note: All dot or config files mentioned below should be placed in the root directory of the project.

The pre-commit framework

Recently, I came to know about the pre-commit framework for managing & maintaining multi-language pre-commit git hooks. This framework itself is written in python and comes along with some pre-commit hooks out of the box. It is configured with a YAML file.

Perform the following steps to install it:

  • Install pre-commit pip install pre-commit .

In .pre-commit-config.yaml, we define the source (repo) from where the hooks will be downloaded, some high-level options and hooks from each source. Below is a sample .pre-commit-config.yaml file that I used in one of the projects.

.pre-commit-config.yaml (pre-commit framework configurations)

Now, we’ll discuss all hooks added in the pipeline one-by-one with its config.

isort: sorts imports

This sorts all the imports in a python file alphabetically and automatically separated into sections. Manually keeping imports sorted alphabetically is tedious when you have so many imports, like in views of Django projects one may have multiple import statements for internal or external packages.

Along with isort we used seed-isort-config hook which statically populates the known_third_part parameter for the isort configuration setting, which otherwise we’ll manually need to populate. Automate everything!
Here’s a sample .isort.cfg file for configuring isort that I used:

.isort.cfg (isort hook configurations)

Black: the code formatter

Black is an uncompromising Python code formatter. I’ve been using it for a long time even without pre-commit hooks, it’s super nice!

DEP 0008 (‘Formatting Code with Black’) was accepted by the Django board. Which proposes to enforce code formatting with Black in Django.

With Black, we get code formatted with PEP 8 compliance, plus some other style choices that the Black team sees as helpful for making maintainable code. Some notable style choices by black are:

  • Unlike in PEP8, characters per line is 88, not 79.

One can always override the default styling choices, Black uses pyproject.toml as a configuration file, which is PEP 518 standard.

Here’s my pyproject.toml file for configuring black:

pyproject.toml (black hook configurations)

Black formats only python files, for other languages you can give a try to prettier used by over million projects on GitHub itself.

flake8: code style checker

flake8 is a wrapper around PyFlakes, pycodestyle and Ned Batchelder’s McCabe script for checking cyclomatic complexity. For black to work along with flake8 we need to keep some configs in sync, like max-line-length and also need to mention some error & warning codes to ignore in the configuration file (.flake8). The error/warning may be from PyFlake error codes or pycodestyle error codes list.
I used the following .flake8 configuration:

.flake8 (flake8 hook configurations)

When added all the above config we good to go! Also, when added these configs to an existing project you may want to format code base for the start, so for that, you just need to run pre-commit run --all-files. ✌️

Conclusion

In this story, I discussed how we can automate the workflow of formatting wrongly formatted python code by creating pre-commit hooks pipeline with various tools/utilities. Though here I was specific with python language, you can use the same steps for other languages by using appropriate linting utility hook for that language. Setting up pre-commit is fairly easy for the team when all configurations are there in the project, they just need to run pre-commit install! Also, it’s better to have these checks also in CI (Continuous Integration) pipeline, because one can always skip these pre-commit checks by passing the flag --no-verify while committing.

Resources

If you liked this story, click the👏 below so that more people will see it here on Medium.

Staqu Dev Logs

While we work on cutting edge products for enterprises and audience at large, this publication sheds some insights into the rapid prototyping, code marathons and optimizations that Staqu’s engineers constantly churn out!

Rohit Gupta

Written by

Engineer ▪️ Pythonista 🐍 ▪️ Explorer ▪️ https://github.com/guptarohit

Staqu Dev Logs

While we work on cutting edge products for enterprises and audience at large, this publication sheds some insights into the rapid prototyping, code marathons and optimizations that Staqu’s engineers constantly churn out!

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade