How to write Flake8 plugins 😍

Automate your code reviews

Martin Thoma
Sep 25, 2020 · 4 min read
Image for post
Image for post
Photo by Photos Hobby on Unsplash

Really important parts of code reviews are almost impossible to automate: Architectural decisions and logical bugs. They are too customized to your codebase; too specific for the pull request.

However, many comments in code reviews are not like that. They are about simply style decisions, common minor mistakes, and misconceptions. They are valuable as well, but they distract the reviewer from the harder parts. The lining system Flake8 allows you to write plugins that automatically capture those simple things. You can execute them in your CI pipeline and thus never need to think about them again.

In this article, you will learn how to create a Flake8 plugin. As an example, we will create a plugin which recognizes the pattern

# Bad
not (a == b)

and thus can suggest using the following instead:

# Good
a != b

About Flake8

Flake8 is a linter which only checks rules. It does not change the code. Every rule has a message and a code. The built-in rule codes begin either with E (Error) or with W (Warning). After that, a 3-digit number follows:

  • E101: indentation contains mixed spaces and tabs
  • E111: indentation is not a multiple of four
  • E112: expected an indented block

One can select the rules which one wants to check on a prefix-basis:

# Check all rules beginning with "E1" and nothing else
flake8 --select E1 .

Alternatively, one can blacklist rules:

# Check all rules except the ones beginning with E1 or W512
flake8 --ignore E1,W512 .

Plugins need a 3-character prefix. For my plugin flake8-simplify, I chose SIM as a prefix.

The 3-character prefix should not start with E or W as people might want to completely block Flake8 W-rules.

The Flake8 Plugin Skeleton

Cookiecutter is a command-line utility that allows you to create a project from scratch by using a template. Install it via

pip install cookiecutter

To create your Flake8 plugin template, use:

cookiecutter https://github.com/MartinThoma/cookiecutter-flake8-plugin

The Flake8 Plugin class

You need to create a plugin class like this:

You need to point to that class in the setup.cfg:

[options.entry_points]
flake8.extension =
AWE=flake8_awesome:Plugin

Just replace AWE by the 3-character code you want to have for your rules. We are, of course, not there jet. The visitor class still needs to be implemented. But to understand the visitor, we need to understand Pythons AST.

Understanding Pythons AST

An Abstract Syntax Tree (AST) is another representation of the code.

Install astpretty to understand how the AST looks like for a piece of code you’re interested in:

pip install astpretty

Here is how you use it:

$ astpretty --no-show-offsets /dev/stdin <<< "not a == b"Module(
body=[
Expr(
value=UnaryOp(
op=Not(),
operand=Compare(
left=Name(id='a', ctx=Load()),
ops=[Eq()],
comparators=[Name(id='b', ctx=Load())],
),
),
),
],
type_ignores=[],
)

Next, we need to recognize that pattern.

A Flake8 visitor class

The logic of the linter is in the visitor class. The visitor gets called for every node in the AST. It has various methods which are based on the operations you see within astpretty, e.g. visit_UnaryOp . This method receives an ast.UnaryOp with which you can do whatever you want. More often than not, this will not be of the pattern you need. In this example, you need to look for the op == Not() and operand==Compare and ops=[Eq()] .

This is how it’s done:

I like astor.to_source to get a string like it was in the source code back from the AST node.

Testing Flake8 plugins

A big shoutout to Anthony Sottile. Before his video (linked below), I had no clue how to test Flake8 plugins. Thank you ❤️

It’s trivial once you know it: You create a string that contains a line or multiple lines of code. You pass it to ast.parse to get the abstract syntax tree. That one can be passed to your plugin, which should generate some output. That output can then be checked:

A non-trivial test looks like this:

Complete example

This is part of flake8-simplify 0.2.0. I’ve pinned the version so that you can see the code in the simple form it has right now.

Please make me proud and automate your reviews — make the following comics true for Flake8 plugins as well :-)

Image for post
Image for post
Comic by geek-and-poke (Oliver Widder)

See also

I can highly recommend the following video by Anthony Sottile (LinkedIn, Github). He is the awesome guy who develops pyupgrade, blacken-docs, and astpretty:

Analytics Vidhya

Analytics Vidhya is a community of Analytics and Data…

Sign up for Analytics Vidhya News Bytes

By Analytics Vidhya

Latest news from Analytics Vidhya on our Hackathons and some of our best articles! Take a look

By signing up, you will create a Medium account if you don’t already have one. Review our Privacy Policy for more information about our privacy practices.

Check your inbox
Medium sent you an email at to complete your subscription.

Martin Thoma

Written by

I’m a Software Engineer with focus on Data Science, Machine Learning. I have over 10 years of experience with Python. https://www.linkedin.com/in/martin-thoma/

Analytics Vidhya

Analytics Vidhya is a community of Analytics and Data Science professionals. We are building the next-gen data science ecosystem https://www.analyticsvidhya.com

Martin Thoma

Written by

I’m a Software Engineer with focus on Data Science, Machine Learning. I have over 10 years of experience with Python. https://www.linkedin.com/in/martin-thoma/

Analytics Vidhya

Analytics Vidhya is a community of Analytics and Data Science professionals. We are building the next-gen data science ecosystem https://www.analyticsvidhya.com

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store