How We Maintain High Levels of Code Quality
For a long time, we used TravisCI and Coveralls for executing lint checkers and tests and tracking our code coverage. These are fine tools but we’ve recently switched to CircleCI and CodeCov. This is our default setup for projects.
Let’s start with a breakdown of our tools. We believe developers should run lints and tests locally during development. Our Continuous Integration (CI) process runs these checks automatically in a clean environment after commits are pushed or a pull request is created. The team-wide check provides an extra level of validation that we are on the right track.
As the mess builds, the productivity of the team continues to decrease, asymptotically approaching zero.
–Robert Martin in What is Clean Code
We prefer the Pinax-style of using of double-quotes over single-quotes. There is plenty of debate over this style preference and I’ll leave the quote holy war out of this post. I’ll just say
flake8-quotes keeps our quote style consistent, which is incredibly awesome.
Next we apply isort.
Just like quotation style, everyone has their own preference when it comes to sorting imports. We use Timothy Crosley’s isort to help maintain consistency. This automation is so nice we revised our import sorting preferences to fit what
isort can do for us.
I usually can very quickly give up on personal taste if the computer can just do it for me.
We maintain tool configurations at the top of each project’s
Then we check our code before committing:
isort --recursive --check-only --diff <package> -sp tox.ini
One great feature of
isort is when you omit the
isort fixes up your imports and saves the changes automatically.
Making tests run fast is important. When a test suite takes too long to complete developers avoid running tests locally. Writing tests efficiently is equally important. If tests are difficult to construct developers shy from writing them.
Our main mission is delivering quality product. Well-tested code is both more robust and simpler to refactor, so we place a high value on the ease of writing on comprehensive test suites. The good news is a solid testing regime actually speeds up development over the life of a product as summarized by our lead quality engineer:
Festina lente, my friend. Take a bit more time, be thorough, plan well, write tests, etc. The end result is faster development.
This Latin motto, festina lente, is more a philosophy. Translated, it means make haste, slowly. This is how we approach crafting software solutions for ourselves and our clients. It absolutely works.
We leverage django-test-plus by Frank Wiles and Revolution Systems to make our test writing faster and more efficient. We use factory_boy to solve a huge headache when building and maintaining test suites in complex projects: the creation of fixture data to drive test scenarios. The combination of
factory_boy substantially reduces test authoring overhead.
To make sure we are testing the things that need to be tested we employ Ned Batchelder‘s popular coverage.py tool. We believe
coverage is a valuable tool, but its utility is sometimes misunderstood, so we wrote a blog post: 5 Reasons You Should Care About Coverage. Read that post if you’re on the fence about using coverage for your projects.
We have config sections in
coverage as well:
Having all these tools running locally is great. But even better is to have robots in the cloud executing lints, tests, and coverage tracing whenever commits get pushed to your repository. This can help catch the human error of forgetting to run some of these tools. It can also catch issues where a local devs environment has gotten out of whack from what’s in source control (e.g pip installed a package the code depends on but didn’t add to
requirements.txt or added a migration but forgot the
We use CircleCI to run our
isort checks and if those pass proceed to running our tests with coverage. Finally, if on
master, we will go ahead an deploy to Eldarion Cloud on successful lints and tests.
This all typically happens within seconds to just a few minutes depending on how complex the test suite is.
We have this process setup to integrate with Github and Slack so that everyone is always up to date. We use Github Flow in our development process and our Pull Requests get annotated with the results from CircleCI (and CodeCov which we will discuss in the next section). This aids in the peer review process letting our engineers focus on what the robots can’t.
Our configuration for CircleCI follows this general pattern:
workflows option in CircleCI adds a lot of power. Basically what’s happening here is:
- We run lint checkers against the code.
- Then execute tests
- If those two items pass, the
deploystep will execute, but only if on the
deploy step executes some shell scripts in our repo that will deploy the code to an instance that we have setup for getting feedback from the product owner as the last step in our development process. What this means in practice is that as soon as a peer review is complete and the pull request is merged, the lints and tests are run and then automatically deployed.
Code Coverage Tracking
We use CodeCov to take the coverage reports uploaded from the CircleCI processing and produce reports to help us keep a pulse on how we are doing coverage-wise. But more important than that and what I think the killer feature of CodeCov is, is the diff reporting.
Each Pull Request is updated with a report on how much the diff is covered in tests. That is, out of the changes submitted by the developer, how much of those changes have been at least executed by a test suite.
This stack of tools and services enables us to remain vigilant about code quality.
By encapsulating a lot of quality metrics into machine readable rules we can let the computer validate the code for us. With tests that can exercise the code in a repeatable fashion, we can move more violently to implement new features or refactor existing code for performance or maintainability.
Finally, this stack helps us to focus on taking ideas to launch, faster, because we are much more able to be focused on the value proposition of the software.
Originally published at eldarion.com on October 17, 2017.