Sharing good practices: linters
Introduction
One of our engineering challenges is to scale our team. We were 30 at the end of 2019. 6 months later, we are 40. And we are all working on the same code base, in the same git mono-repo.
As we grow, we have more and more challenges sharing what we consider “good practices”. It is a challenge for current engineers. We need to make sure they follow the new rules we put in place. It is also challenging for newcomers. They arrive with their habits, and we need to show them how we do software engineering.
The question is how can we continue to grow our team while, at the same time, increasing our level of quality, and without slowing down our velocity.
Side note, I will not go into details on what is a “good practice”, there are enough trolls articles on the internet about them. I define a “good practice” as an habit a team has at a certain point in time. It is something that can change from one company to another, and over time.
Sharing practices
We have a strong culture of written communication. So, when a newcomer joins our engineering community they are encouraged to read our wiki and documentation. They are also invited to internal technical trainings. It’s a great way to learn practices that are active at one point in time.
For now it’s quite successful. But it has limitations.
How do you bring new practices? How do you make sure everyone is following them?
We have documentation, we communicate on changes. We also miss notifications, and go on vacations, and we forget.
For example, on security. We have security practices in place. On the other end, changing code linked to security is not something you do every day. And it is easy to forget something when you have read it 6 months before.
Another exaggerated example is: tabs vs spaces. We know each developer has its own width set of formatting rules.
For all those reasons, we decided to use automatic tooling to help us as much as possible. We are using black for formatting, and various flake8 linters for automatic checks for our Python code. For our JavaScript and TypeScript code, we use eslint and prettier. As mentioned above, our “best practices” are not yours. The configuration of those tools are not what really matters. What is important is that they are static checks. Checks that are run automatically and helps our engineers to scan for our good practices.
This is not something new. Most teams have similar tools in place.
Of course linters are not perfect, and we allow ourselves to disable them sporadically. Each time engineers disable a linter, they are encouraged to explain why they did it. Next time someone will browse this specific code they will be able to learn something or challenge it if it’s not relevant anymore, thanks to this small comment.
Those tools provide practices that are what the community of a given language consider as good ones. And again, those are not what we consider “good”. We also have our own set of rules that we want to follow that are not language specific. And we wanted to have a way to enforce those rules.
Custom linters
As we looked for solutions to apply those practices we have, the internet told us “linters”. I have always considered linters to be a very advanced topic of engineering. Something only a few gifted people were able to code. I was reluctant to code anything related to this. It appears I was quite wrong.
We stumbled upon the local plugins features of flake8. This feature exists since 2017 and it enables you to run custom Python code as a fully fledged linter. The documentation is not perfect, but after a few trials and errors, we have managed to code one check: to make sure the decorator @dataclass_json is defined before the decorator @dataclass(defining them the other way around results in runtime bugs). Once we had a working example, we’ve managed to add more of them.
A few weeks later, we also looked for solutions for eslint. And as for flake8, it was also simple to add custom rules in eslint. This time we added a deprecation for CSS class names that shouldn’t be used anymore.
Some code
For flake8
If you are interested in the technical part, a flake8 linter is a class that traverses the Python AST and yields an error if needed. All our simple linters have the same structure. Here is an example of a linter that forbids the usage of classes. Disclaimer, we do not use it 😉.
In your .flake8 file add this section:
[flake8:local-plugins]extension =ALAN001 = tools.flake8.no_classes:NoClassesLinter, #the Python path to your class
In a file called tools/flake8/no_classes.py:
import astclass NoClassesLinter:
def __init__(self, tree, filename):
# we just save those as object properties to use them later
self.tree = tree
self.filename = filename
def run(self):
# we traverse the AST of the pytchon code from the file "filename"
for node in ast.walk(self.tree):
# if we have a declaration of a class
if isinstance(node, ast.ClassDef):
# we yield an error
# an error is:
# * the line number where the error is
# * the column where the error is
# * a string representing the error (user facing)
# * the linter that produced the error
yield (
Node.lineno,
node.col_offset,
"ALAN042 classes are forbidden",
self,
)
Next time you will run flake8, the linter will also run.
For ESLint
An ESLint plugin is a simple node package, with a name starting with eslint-plugin-. Let’s say you want to create a plugin with a rule that forbids the usage of JavaScript ES6 classes.
You can create a node package called “eslint-plugin-myplugin”, that you can install or link in your current project.
The package has to expose an object, with a rules field that matches a rule name to a function that takes a single object, called the “context”.
The rule function returns an object matching AST node types to functions. Each can analyze the node as it pleases, and report errors using the contenxt.report function.
Here’s what it could look like:
module.exports = {
rules: {
"no-classes": function noClasses(context) {
return {
ClassDeclaration(node) {
context.report({
node: node,
message: "no classes allowed"
});
}
};
}
}
};Once you install or link the plugin package in your main project (with npm install or npm link), you can add its name to your .eslintrc file, and add the rule to the rules list:
{ "plugins": ["myplugin"],
"rules": {
"myplugin/no-classes": "error"
}}
You can find more details in the documentation of eslint.
Conclusion
We now have seven custom Python linters that match our practices, and one for JavaScript. All of them are specific to us, and not really worth sharing (like deprecated usage of some global variables).
We now use them on a daily basis, and it helps us push new practices one module at a time.
