How We Maintain High Levels of Code Quality

For a long time, we used TravisCI and Coveralls for executing lint checkers and tests and tracking our code coverage. These are fine tools but we’ve recently switched to CircleCI and CodeCov. This is our default setup for projects.

Patrick Altman
Oct 17, 2017 · 5 min read
Image for post
Image for post
Photo by on

Let’s start with a breakdown of our tools. We believe developers should run lints and tests locally during development. Our Continuous Integration (CI) process runs these checks automatically in a clean environment after commits are pushed or a pull request is created. The team-wide check provides an extra level of validation that we are on the right track.


Our tools ensure code stays . is crucial for long term productivity.

As the mess builds, the productivity of the team continues to decrease, asymptotically approaching zero.

–Robert Martin in

We start with using the plugin.

We prefer the Pinax-style of using of double-quotes over single-quotes. There is plenty of debate over this style preference and I’ll leave the quote holy war out of this post. I’ll just say flake8-quotes keeps our quote style consistent, which is incredibly awesome.

Next we apply .

Just like quotation style, everyone has their own preference when it comes to sorting imports. We use Timothy Crosley’s to help maintain consistency. This automation is so nice we revised our import sorting preferences to fit what isort can do for us.

I usually can very quickly give up on personal taste if the computer can just do it for me.

We maintain tool configurations at the top of each project’s tox.ini file:

Then we check our code before committing:

flake8 <package>
isort --recursive --check-only --diff <package> -sp tox.ini

One great feature of isort is when you omit the --check-only flag, isort fixes up your imports and saves the changes automatically.

Robots FTW!


Making tests run fast is important. When a test suite takes too long to complete developers avoid running tests locally. Writing tests efficiently is equally important. If tests are difficult to construct developers shy from writing them.

Our main mission is delivering quality product. Well-tested code is both more robust and simpler to refactor, so we place a high value on the ease of writing on comprehensive test suites. The good news is a solid testing regime actually speeds up development over the life of a product as summarized by our lead quality engineer:

, my friend. Take a bit more time, be thorough, plan well, write tests, etc. The end result is faster development.

This Latin motto, festina lente, is more a philosophy. Translated, it means make haste, slowly. This is how we approach crafting software solutions for ourselves and our clients. It absolutely works.

We leverage by and to make our test writing faster and more efficient. We use to solve a huge headache when building and maintaining test suites in complex projects: the creation of fixture data to drive test scenarios. The combination of django-test-plus and factory_boy substantially reduces test authoring overhead.


To make sure we are testing the things that need to be tested we employ ‘s popular tool. We believe coverage is a valuable tool, but its utility is sometimes misunderstood, so we wrote a blog post: . Read that post if you’re on the fence about using coverage for your projects.

We have config sections in tox.ini for coverage as well:


Having all these tools running locally is great. But even better is to have robots in the cloud executing lints, tests, and coverage tracing whenever commits get pushed to your repository. This can help catch the human error of forgetting to run some of these tools. It can also catch issues where a local devs environment has gotten out of whack from what’s in source control (e.g pip installed a package the code depends on but didn’t add to requirements.txt or added a migration but forgot the git add).

Continuous Integration

We use to run our flake8 and isort checks and if those pass proceed to running our tests with coverage. Finally, if on master, we will go ahead an deploy to on successful lints and tests.

This all typically happens within seconds to just a few minutes depending on how complex the test suite is.

We have this process setup to integrate with and so that everyone is always up to date. We use in our development process and our Pull Requests get annotated with the results from CircleCI (and CodeCov which we will discuss in the next section). This aids in the peer review process letting our engineers focus on what the robots can’t.

Our configuration for CircleCI follows this general pattern:

The workflows option in CircleCI adds a lot of power. Basically what’s happening here is:

  1. We run lint checkers against the code.
  2. Then execute tests
  3. If those two items pass, the deploy step will execute, but only if on the master branch.

The deploy step executes some shell scripts in our repo that will deploy the code to an instance that we have setup for getting feedback from the product owner as the last step in our development process. What this means in practice is that as soon as a peer review is complete and the pull request is merged, the lints and tests are run and then automatically deployed.

Code Coverage Tracking

We use to take the coverage reports uploaded from the CircleCI processing and produce reports to help us keep a pulse on how we are doing coverage-wise. But more important than that and what I think the killer feature of CodeCov is, is the diff reporting.

Each Pull Request is updated with a report on how much the diff is covered in tests. That is, out of the changes submitted by the developer, how much of those changes have been at least executed by a test suite.


This stack of tools and services enables us to remain vigilant about code quality.

By encapsulating a lot of quality metrics into machine readable rules we can let the computer validate the code for us. With tests that can exercise the code in a repeatable fashion, we can move more violently to implement new features or refactor existing code for performance or maintainability.

Finally, this stack helps us to focus on taking ideas to launch, faster, because we are much more able to be focused on the value proposition of the software.

Originally published at on October 17, 2017.


Taking ideas to launch, faster, since 2009

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch

Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore

Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store