7 Drawbacks Of Linting Tools For Static Analysis

Linting tools (also known as linters or static analyzers) help automate the code review process. They perform basic static code analysis by flagging programming errors, bugs, style issues and security vulnerabilities before code is compiled and actually runs. The overall aim is to improve software quality and reduce costs.

While integrating these tools into your workflow is largely beneficial, some drawbacks exist which may not make them the best choice for long term or more sophisticated software development.

1. Code Checks Limited To One Programming Language

Many free, open source linters perform code checks on only one coding language. This includes popular linters (per coding language) below:

• Python: Pylint, Pyflakes, Flake8 (read more)
• Java: Checkstyle, PMD, FindBugs (read more)
• Ruby: Brakemen, rust-clippy, Rubocop (read more)
• Scala: ScalaStyle, wart remover (read more)
• JavaScript: ESLint, JSLint

These thirteen linters exemplify choices for less than 1% of coding languages that exist (sources like Wikipedia count more than 700). While difficult to pinpoint the exact number of linting tools, we know that there are hundreds since just the top 189(!) have been identified by AwesomeOpenSource.com.

Given such variety, technology stacks with multiple coding languages may need multiple tools integrated into the development workflow. This may produce negative effects which we describe in more detail below.

2. Variable Functionality For A Narrow View Of Code Quality

Similarly, the variety of functions among tools makes standardization difficult. Some focus on just one aspect of code quality like security, syntax, code coverage or code style. For instance, among Python static analysis tools, Pyflakes analyzes code syntax whereas Flake8, another tool, is used for stylistic checks.

With multiple tools performing a variety of functions it is difficult to obtain an integrated, holistic view of code quality. Also, it is bothersome to have the same code errors flagged by multiple tools.

3. Difficulty Handling Complex Rules

Linters analyze code for stylistic and programming errors against the rules they already know. While great for identifying errors using standard rules, this may pose problems for more unconventional code practices.

Although you may be able to avoid errors by overriding defaults and adjusting or disabling rules for particular files, this could cost developers’ time.

4. False Positives

A common critique of linters is that they produce many false positives. In other words, they flag issues that are not true issues in source code. While most developers find false positives annoying some use them as a sign to adjust rules for particular files.

However, adjusting rules becomes complicated when configuring multiple linters. Each linter has a different configuration file (with different syntaxes and semantics) that also needs to be adjusted. Therefore, in order to disable a rule considered false positive, developers should read the documentation of each linter before tweaking the file.

5. Reporting On Only Basic Metrics

While linting tools are good for basic metrics, they may not be the best choice to report on more complex indicators of software quality. This includes cyclomatic complexity which gauges the complexity of a program. Also, it is key to integrate linting tools early in the development workflow so that reporting and incorporating code quality metrics occurs before production.

6. Limited Learnings

While they identify best practices, linters do not ensure teachings of best practices. Developers can use linting tools to improve code, but they might not be able to replicate the best practices.

7. Hidden Costs

While open source tools seem “free”, hidden costs related to many of their mentioned shortcomings exist. For instance, integrating multiple tools in the development workflow requires maintenance and monitoring of each tool. When new updates occur, changes will need to be implemented.

This additional workload may fall on the plate of development or DevOps teams who could otherwise spend their time building new features.

Code Review Suites

Fortunately, advanced static analysis tools exist which mitigate most drawbacks of using individual linters. Code review suites integrate some of the most well known linting tools on one platform. This ensures the detection of problems across languages.

For instance, Codacy’s code review suite integrates several linting tools plus its own rules in a continuous manner ensuring that tools are updated and new, useful tools are introduced. This helps support 30+ programming languages using one tool for code quality automation. Also, rather than needing to read documentation in order to disable a rule considered false positive, users can click on one button to disable a rule on Codacy.

While tools like Codacy may come at a monthly cost, for some use cases it may be worth it.

Overall linters are a strong tactic for automated code review and important to incorporate into your workflow. However, keep in mind the downsides before you incorporate too many of them as this may actually add costs rather than reduce them.

Originally published at https://blog.codacy.com on February 5, 2020.