PMD meets Git (Part II)

In our previous post, we described how the git blame command could be useful to prioritize technical debt reported by PMD. In short, Git knows the date and the author of each line of code, whereas PMD reports if a line contains coding style issues. Crossing Git and PMD, specially in huge projects, helps us to identify hotspots of our project according our current development priorities.

Developers usually decide to use tools such as PMD to measure the accumulated technical debt when they experiment serious productivity problems. However, static code analysis tools generally do not offer an incremental execution to only analyze our last modifications. In other words, we are not able to resolve automatically if we are introducing new coding style issues before committing.

Getting feedback from static code analysis tools after committing and pushing our changes produces a waste of resources.

Developers should be able to identify and fix coding style issues without polluting the Git history, running unnecessary CI jobs(e.g Jenkins) and more importantly, before changing their focus to another coding task.

Moreover, static code analysis tools should be integrated in many places of our development workflow, which means automatic comments our code review or post validations of our CI jobs, but these are only applicable as a preventive solution for rebel developers.

The habitual Git workflow we have found for continuous code inspection consists of:

1. Having a reference branch (e.g master) where everything works.
2. Developers creates a branch per feature to implement.
3. After verifying that this branch is not introducing extra coding style issues (and everything else is OK), it is merged into the reference branch; preferably as a single commit (git squash).

Git could facilitate incremental static code analysis, too.

As you know, the git diff command identifies which are the modified lines (i.e not committed or pushed) of our source code and the corresponding mapping of the line numbers of a previous version and the current one.

Obviously, those issues located in the modified parts of our code, are new. However, there are some coding style issues like “unused variable” that may point to unmodified regions of code, but are a consequence of removing the references to that symbol. For example, in the following snippet, the boolean variable “find”, becomes unused after removing our loop.

git diff example output

In order to identify which are the new issues of untouched parts of our code it is necessary to compare them with those reported issues during our last PMD analysis.

It is important to use the concept of last analysis, which resolves the last analyzed commit and the corresponding issues, to be able to reproduce the same results from our local machine, which still may have pending changes to commit or push, than from continuous integration tools.

In order to evaluate which is the corresponding commit of our last analysis and the corresponding issues of the affected files, we have designed a REST API in WalkModHub.

GET /analysis/:repo/:branch
GET /analysis/issues/:repo/:branch?location=:file

Notice after retrieving the issues of our last analysis, it is necessary to use Git to compare two file versions and identify the new line numbers of previously reported issues.

The easiest way to validate this approach by extending PMD with a new Renderer component. Renderers are notified every time PMD resolves the issues for an specific file. The principal Renderer method to implement is renderFileViolations.

public void renderFileViolations(Iterator<RuleViolation> violations) throws IOException

Our incremental renderer implementation uses JGit to resolve the file status and the line number of those previously reported issues. The corresponding source code is here. You may argue that at a design level, it violates the single responsibility principle because we are coupling the filtering logic with the rendering. However, this is our first attempt to validate our incremental approach before adding excessive changes to the official PMD project.

Now, are you ready to play with our new PMD renderer? You just need to follow the next steps.

1. Download and run the last WalkModHub Docker compose.

wget -O docker-compose.yml http://tinyurl.com/j23y8g6
docker-compose up

2. Open our browser in localhost or in our Docker machine IP

http://localhost:80

3. Run your first analysis for a Git repository (e.g https://github.com/rpau/test.git), which will give us a repository ID. For this preliminary version of our software, the reference Git branch is always master.

4. Download and install our modified PMD from here. Our next steps will assume that you use the pmd command as an alias for bin/run.sh of this binary distribution.

5. Run pmd from your local copy of your Git repository with the following instruction. Replace ${WH_HOST} for your WalkModHub IP address (e.g localhost) and ${REPO_ID} for your own repository ID.

pmd pmd -d . -f WalkModHub -P host=${WH_HOST} -R http://${WH_HOST}/pmd/${REPO_ID}/master/ruleset.xml

And voilà! If there are no changes to your repository, PMD will not report any new issues. Otherwise, you will only see those that have been produced by your own changes.

For those which are not familiar with the PMD arguments, we are specifying our renderer with the -f (format) parameter. The -P arguments define properties for our render; and in this case, we are defining where the renderer can lookup the last analysis of our repository. Finally, the -R (ruleset) argument specifies the set of rules to take in consideration.

WalkModHub generates the ruleset config according the quality gate for that project, which is specially useful for external developments. However, if your project already have their own PMD ruleset.xml configuration, because it is an agreement between developers, everything will be synchronized with that.

Probably, if you are a Java developer, you would like to use this PMD version from a building tool (e.g Maven or Gradle) plugin. Before our contribution becomes officially accepted by PMD, the most appropriate way to proceed is to install our PMD version locally and run it from plugins for command line tools. For example, the snippet to add in our build.gradle file, should be:

task pmd(type:Exec) {
workingDir '.'
commandLine 'pmd', 'pmd', '-d', '.', '-f', 'WalkModHub', '-P', 'host=${WH_HOST}', '-R', 'http://${WH_HOST}/pmd/${REPO_ID}/master/ruleset.xml'
}

Then, after running

gradle pmd

The system prints ONLY our new issues.