ESLint Custom Tooling at Scale

Cheng Gibson

Published in

SafetyCulture Engineering

11 min readFeb 21, 2022

Context

SafetyCulture offers iAuditor as a web and mobile Software-as-a-Service (ie. SaaS) to our customers.

Our web SaaS code monorepo uses TypeScript, React, Redux, and ESLint, amongst other tools.

We use ESLint to ensure high code quality and to promote coding best practices, via the usage of plugins and rules such as:

eslint-config-airbnb (promotes industry code conventions)
eslint-plugin-jsx-a11y (promotes web accessibility best practices)
eslint-plugin-react-hooks (ensures correct usage of React Functional Components).

However, as our team sizes grew over time, we faced difficulties in enforcing internal code conventions, as manual code reviews are not scalable and are error-prone.

We then investigated ways to automate manual code reviews using static code analysis tooling.

What is Linting / ESLint?

Linting is the usage of static code analysis tools to detect and enforce specific patterns in the analyzed code.

ESLint is a linter that is commonly used industry-wide for JavaScript/TypeScript codebases, and is used by Microsoft, Facebook, Airbnb, Netfix, Box, Paypal, etc. (see more). It is also used by many open source projects such as React, NextJS, ExpressJS, and Yarn (see more).

The ESLint ecosystem contains many useful rules and plugins that can help to ensure high code standards in a codebase. (See: awesome-eslint)

Problem / Solution in Brief

Problem

I. Difficult to enforce coding standards at scale. (ie. manual code reviews are not scalable and error-prone)

II. Hard to introduce new ESLint rules incrementally. (ie. if a new rule flags a lot of errors, one would need to fix many files at a go before one can enable the rule)

Solution

I. Use ESLint Custom Plugins

II. Use eslint-workflows CLI

Read on to find out more about our solutions 🙂

Solution I: ESLint Custom Plugins

In Brief

ESLint’s plugin system allows us to write custom code to detect and enforce code standards, which reduces the need for manual code reviews and keeps our codebase’s code quality high.

We gathered feedback from our internal developer community, identified areas of difficulty with their developer experience journey, and developed custom ESLint rules to solve them.

Areas of Application

✅ Prevent runtime bugs

Rule: @safetyculture/rules-of-defineroutes (source)

Example: `@safetyculture/rules-of-defineroutes VSCode lint error`

`Example: @safetyculture/rules-of-defineroutes CLI lint error`

Problem:

We use defineRoutes() to specify web URL entrypoints to our web app. (which is a light wrapper around React Router <Route/> declarations)
Developers can declare new entries that accidentally clobber old routes, which results in breaking existing user flows.
In the given example, the /user/:id/settings route has been clobbered by /user/:id , since for the given URL /user/123/settings , it would be matched with the first entry, and thus would never match with the second entry.

Solution:

Write custom ESLint rule: detects clobbered paths, duplicated paths, paths with invalid type (eg. non-string literal, for more effective static analysis)

2. ✅ Aid with code migrations

Rule: @safetyculture/require-i18n-context (source)

Example: `@safetyculture/require-i18n-context VSCode lint error`

Problem:

We use translation services to provide localisation support for our iAuditor SaaS product. (See more: Localisation at SafetyCulture)
Lack of context for translated strings → Inaccurate translations → Poor user experience

Solution:

Write custom ESLint rule: enforces usage of localisation string tags (ie. i18n.tand <Trans/>) with context.
ie. if no string context provided, raise lint error

3. ✅ Promote internal code conventions

Rule: @safetyculture/no-render-import-testing-library (source)

Example: `@safetyculture/no-render-import-testing-library VSCode lint error`

Problem:

When writing frontend tests, developers may choose either to use render or initRender() from @safetyculture/testing-library (entrypoint for our internal testing library)
According to our internal convention, developers should always use initRender() over render (except in valid use cases).

Solution:

Write custom ESLint rule: Detects illegal usage of render
ie. if render is imported from library, raise lint error

4. ✅ Provide useful suggestions for improved developer experience

Rule: @safetyculture/sc-packages-import-suggestion (source)

Example: `@safetyculture/sc-packages-import-suggestion VSCode lint error`

Problem:

We have internal common libraries that are used across different domains.
VSCode import suggestions always provides not-quite-correct suggestions for our internal monorepo imports. (eg. packages/sc-app-state)

Solution:

Write custom ESLint rule: Suggest useful fix for any import with packages/*
eg. packages/sc-app-state → @safetyculture/sc-app-state
This plugin helps save developer’s keystrokes by automating away this manual and common task, thus improving developer experience.

How to Write an ESLint Plugin

Thanks to useful community tooling, it’s easy to get started with your own ESLint plugin and rules 🙂

Conceptualize exact pattern(s) that you would like to detect.

eg. Enforce naming conventions for React components? (ie. must not contain ‘component’ inside name)

eg. Disallow test blocks that are longer than 20 lines?

2. Create test cases for OK / Not OK code

Since code patterns can vary wildly, having a rigorous set of test cases helps to ensure that your plugin behaves correctly in different scenarios.

// OK
it('passes', () => { ... })// Not OK, test block exceeds 20 lines
it('passes', () => { ...
  // 20 lines here
})

3. Use AST Explorer to view AST examples

AST Explorer allows us to quickly reason about how our code look like from an Abstract Syntax Tree (ie. AST) point of view.

You can use AST Explorer to experiment with different code patterns to get an idea of the different data structures that you will have to handle in your rule code.

Tip: Don’t forget to set the correct language (eg. JavaScript) and parser (eg. @typescript-eslint/parser) according to your target platform.

4. Create ESLint plugin and rules scaffolding

Install generator-eslint, which is used to generate code scaffolding for new ESLint plugins.

Use the yo eslint:plugincommand to generate a new ESLint plugin. (eg. eslint-plugin-mycompany)

Use the yo eslint:rule command to generate a new rule within your plugin. (eg. mycompany/max-test-block-lines)

5. Implement ESLint rule logic

ESLint uses the Visitor Pattern internally as it traverses the code’s AST nodes — selectors (eg. CallExpression) will be the entrypoints for our code.

ie. Given that a node has matched a selector, ESLint will invoke our custom function for further processing.

// .../lib/rules/max-test-block-lines.jscreate(context) {
  return {
    CallExpression: (node) => {
      if (node.callee.name !== "it") {
        return;
      }const { start, end } = node.loc;
      const exceedsMaxLength = end.line - start.line > 20;
      if (!exceedsMaxLength) {
        return;
      }context.report({
        node,
        message: "Test block exceeds 20 lines.",
      });
    },
  };
}

In the example above, we perform the following:

If node is of type CallExpression, invoke function.
If function name is not equals to it , skip the node.
If node loc does not span more than 20 lines, skip the node.
Else, report the node as a lint error, and affix an appropriate error message.

Useful links:

Working with Rules — ESLint
Working with Plugins — ESLint
Selectors — ESLint
ECMAScript® 2016 Language Specification (useful for looking up technical terms)

6. Fill out test cases

The rule’s test suite is your guarantee that your rule is behaving as per expected.

Copy the test cases defined in Step 2 into the test suite for your rule.

// .../tests/lib/rules/max-test-block-lines.jsruleTester.run("max-test-block-lines", rule, {
  valid: [`it('passes', () => {});`],
  invalid: [
    {
      code: `it('passes', () => {${"\n".repeat(21)}});`,
      errors: [
        { message: "Test block exceeds 20 lines.", type: "CallExpression" },
      ],
    },
  ],
});

7. Use the ESLint rule and verify behavior

Use the rule by defining the following in your eslint config:

"rules" : {
   // your other rules
   "mycompany/max-test-block-lines": "error"
}

Then, run eslint and verify that the lint error shows up:

Example: `@safetyculture/max-test-block-lines VSCode lint error`

`Example: @safetyculture/max-test-block-lines CLI lint error`

Solution II: eslint-workflows CLI

Problem

When introducing a new rule to a codebase, oftentimes it raises a lot of lint warn and error messages.

Generally, there are 3 ways to handle this scenario:

Introduce rule at warn level, then fix lint warnings (creates a lot of noise in ESLint output).
Fix all lint errors at one go, then introduce rule rule at error level (risky from a Root Cause Analysis point of view, as big bang approach introduces a lot of changes in a single go).
Introduce rule at error level, then use config overrides to ignore all files, then incrementally remove overrides by folder/file and fix lint errors (very tedious and manual).

Most of these approaches either require too much manual work, are error-prone, and/or do not prevent new code with lint problems from being added to the codebase.

Solution in Brief

Example: eslint-workflows usage — add-rule

eslint-workflows achieves the following goals:

✅ Allows developers to introduce rules to the codebase without having to fix all lint errors at the start
✅ Allows developers to incrementally fix code with lint errors
✅ Prevents new files with lint errors from being added to the codebase
✅ Uses rcfile-teams.yml to store state of overrides, and is also a source of truth in code form (as opposed to manual tracking)
✅ Integrates with Github CODEOWNERS for easy tracking of entries in rcfile-teams.yml

Technical Approach

Integration with ESLint

eslint-workflows is a further enhancement on Approach 3 — Use overrides to tell ESLint while specific files to ignore lint errors.
The state of overrides is stored in a rcfile-teams.yml file.
When eslint is invoked, overridesentries are generated based off the yml file’s contents.

// .eslintrc.js
const { makeTeamExceptions } = require('eslint-workflows');module.exports = {
  ...
  overrides: [
    // generates overrides based off yml file
    ...makeTeamExceptions(),
  ],
};

CLI Commands

eslint-workflows supports the following commands:

eslint-workflows add-rule: Adds a rule entry to the yml file.

ie. silences lint errors for a given rule, and also detects which specific files to provide overrides for (based off ESLint JSON output + GitHub CODEOWNERS).

2. eslint-workflows remove-rule: Removes a rule entry from the yml file.

3. eslint-workflows remove-rule-folder: Removes a folder from a rule entry from the yml file.

rcfile-teams.yml Schema

The yml schema was designed with the following goals in mind:

Easy to tell which team has which overrides
Easy to tell how many files/folders have overrides
Allows overrides for multiple rules at the same time
Machine-readable

yml file Example:

// rcfile-teams.yml
react-hooks/rules-of-hooks:
  teams:
    "@SafetyCulture/alpha":
      - /packages/alpha/issue-detail.tsx
    "@SafetyCulture/omega":
      - /packages/omega/user-page.tsx
      - /packages/omega-common/utils.ts

Therefore, from this example, we can derive the following:

For the eslint rule react-hooks/rules-of-hooks, there are two teams that need to look into files/folders for this lint error
For the @SafetyCulture/alpha team, they have the following files/folders that contain lint errors: /packages/alpha
For the @SafetyCulture/omega team, they have the following files/folders that contain lint errors: /packages/omega/ , packages/omega-common/

Wins

Used in conjunction with @safetyculture/require-i18n-context, the Localisation team was able to effectively enforce context strings to the entire codebase without having to worry about new code not having context strings.
Developers have been able to add new ESLint rules without having to pollute the ESLint output (ie. many warn and error lint output)

Challenges & Lessons

Not everything is statically analyzable

Code patterns can vary a lot, and depending on your scenario, static analysis may not be powerful enough for you to achieve your goal.

Example: Need to check if test case name starts with a lowercase letter.

// ok
it('passes', () => {})// ok
it('passes' + 'also', () => {})// not ok, static analysis cannot evaluate variable value 
// (only known at runtime)
it(process.env.ENV + ": passes", () => {})

One way to make your code more amenable to static analysis is to constrain the grammar (ie. to restrict the code patterns that are permissible).

// throw lint error, only allow string literal for first argument of it()
// alternative: do not process expressions
it(process.env.ENV + ": passes", () => {})

2. ESLint plugin code can get messy, so test your plugins rigorously

As ESLint code is often written from a perspective of handling AST nodes, the resulting code can often be unintuitive (as opposed to more commonplace CRUD code)
Therefore, it is advantageous that you make your plugin’s test suite as rigorous as reasonably possible to cover your bases.
Additionally, it would also make it easier for future developers to understand your code and to make changes to your plugin (ie. self-documenting code).

3. ASTs can vary a lot, so code defensively

Whilst developing your rule’s code, oftentimes your rule can cause ESLint to crash due to faulty assumptions on the structure and properties of a given node (eg. trying to access a node’s property which may be undefined for other node types).
Additionally, ASTs for most real-world code are oftentimes complex and deeply-nested, so there are a lot of ways that things can go wrong.
Thus, you should code defensively to ensure that your plugin doesn’t crash. (eg. check that node property exists before accessing it, or access property via lodash _.get())

4. Know which parser you are targeting

ESLint works with a variety of parsers to support JavaScript, as well as other closely related variants such as TypeScript and Flow (eg. @babel/eslint-parser, @typescript-eslint/parser, @babel/preset-flow)
Select the correct parser as per your toolchain. (eg. if your codebase is written in TypeScript, you should use @typescript-eslint/parser).

5. Don’t forget to take others along the journey

Static code analysis can seem like black magic to outsiders looking in — it is understandable that others unfamiliar with it would dismiss it as too esoteric or irrelevant to their day-to-day work.
A good way raise awareness of how static code analysis is useful is to show examples of how it helps to solve developer problems or it improves developer experience.
Start the conversation within your local Frontend Guild: Present this at your guild meetings, open a RFC, create a Wiki page. Get the ball rolling! ⚽️
You may also use the 4 Areas of Application as a seed or rough guideline to group your custom ESLint rule ideas before pursuing them.

In Summary

Custom ESLint plugins has allowed us to improve our platform’s quality in an automated manner via the usage of static code analysis to achieve the following:

✅ Prevent runtime bugs
✅ Aid with code migrations
✅ Promote internal code conventions
✅ Provide useful suggestions for improved developer experience

“Custom ESLint plugins represent a large opportunity for organizations to enforce internal code standards and internal best practices at scale”

eslint-workflows has allowed us to introduce new ESLint rules and fix lint errors incrementally, all whilst preventing new code with lint errors from being added to the codebase.

“In the same way that we make testing easy so that we have better code quality, we should also make it easy for developers to introduce new ESLint rules, so that it is easy for developers to improve the overall codebase quality”

Alas, my friends, this is where we part — All the best with your ESLint rule spelunking journey, voyager 🙂

ESLint Custom Tooling at Scale

Context

What is Linting / ESLint?

Problem / Solution in Brief

Problem

Solution

Solution I: ESLint Custom Plugins

In Brief

Areas of Application

How to Write an ESLint Plugin

Solution II: eslint-workflows CLI

Problem

Solution in Brief

Technical Approach

Wins

Challenges & Lessons

In Summary

Written by Cheng Gibson