Overview of your dependencies in your software project

Motivation

Often a requirement in software projects is to list all your open-source dependencies of your code with licenses being used including the source code url, version number and name of the library. The reasons can be different — one reason could be that your customer wants to have this list to make a approval for your software or you want a documentation of libraries being used.

The task to create such lists is time-consuming and error-prone if you do it manually. Additionally, you always have to update the list for a new version of your software.

So let us see if this task can be automated.

Licence Report Generators

When you look around there are many license report generators available, mostly depending on the buildsystem you use.

In our case we have an application which uses npm as dependency management for frontend and gradle as build tool for the backend.

Example — npm based projects

Our project based on npm defines the dependencies via the file package.json. These entries in the json path dependencies are of our interest.

So let us start with a simple licence checker, the nlf project. nlf is a project which helps you to list all licenses in your project.

Install it with npm like this:

npm install -g nlf

You can now run nlf via commandline in your project folder:

nlf -d

This command will generate a list of dependencies with the license found in the package.json file (-d will exclude the developer dependencies).

The output (shortened) looks like this:

ajv@5.2.2 [license(s): MIT]
├── package.json:  MIT
└── license files: MIT
angular@1.6.5 [license(s): MIT]
├── package.json:  MIT
├── license files: MIT
└── readme files: MIT
angular-animate@1.6.5 [license(s): MIT]
├── package.json:  MIT
├── license files: MIT
└── readme files: MIT
angular-aria@1.6.5 [license(s): MIT]
├── package.json:  MIT
├── license files: MIT
└── readme files: MIT
angular-cookies@1.6.5 [license(s): MIT]
├── package.json:  MIT
├── license files: MIT
└── readme files: MIT
angular-file-saver@1.1.3 [license(s): MIT]
├── package.json:  MIT
└── readme files: MIT
angular-material@1.1.4 [license(s): MIT]
├── package.json:  MIT
└── license files: MIT
angular-sanitize@1.6.5 [license(s): MIT]
├── package.json:  MIT
├── license files: MIT
└── readme files: MIT
angular-toastr@2.1.1 [license(s): MIT]
├── package.json:  MIT
└── license files: MIT
angular-touch@1.6.5 [license(s): MIT]
├── package.json:  MIT
├── license files: MIT
└── readme files: MIT
angular-translate@2.15.2 [license(s): MIT]
├── package.json:  MIT
└── license files: MIT
angular-translate-handler-log@2.15.2 [license(s): MIT]
└── package.json:  MIT
angular-translate-loader-static-files@2.15.2 [license(s): MIT]
└── package.json:  MIT
angular-truncate@0.1.2 [license(s): MIT]
├── package.json:  MIT
└── license files: MIT
angular-ui-router@0.3.2 [license(s): MIT]
├── package.json:  MIT
└── license files: MIT
angular-ui-router-title@0.1.1 [license(s): MIT]
├── package.json:  MIT
└── readme files: MIT
angular-ui-tree@2.22.5 [license(s): MIT]
├── package.json:  MIT
└── license files: MIT
angular-xeditable@0.8.0 [license(s): MIT]
├── package.json:  MIT
└── license files: MIT
animate.css@3.5.2 [license(s): MIT]
├── package.json:  MIT
├── license files: MIT
└── readme files: MIT
good-listener@1.2.2 [license(s): MIT]
└── package.json:  MIT
groupdms-admin-frontend@1.0.0 [license(s): Unknown]
jquery@3.2.1 [license(s): MIT]
└── package.json:  MIT
jsnlog@2.24.0 [license(s): MIT]
├── package.json:  MIT
└── license files: MIT
json-schema-traverse@0.3.1 [license(s): MIT]
├── package.json:  MIT
└── license files: MIT
json-stable-stringify@1.0.1 [license(s): MIT]
├── package.json:  MIT
├── license files: MIT
└── readme files: MIT
LICENSES: Apache-2.0, MIT, Public Domain, Unknown

Hint: Unknown comes from the project itself.

As you can see, we only use “easy to use”-licenses like Apache-2.0, MIT and Public Domain in our project which will make approvals very easy.

If this is not enough, we can use the licence-checker project to generate a richer output with more metadata.

license-checker — production — csv — customPath licenceCheckerFormat.json

license-checker will generate of all production dependencies (no dev-dependencies) in csv format.

The fields are being used can be configured with the licenceCheckerFormat.json file which looks like this:

{
        "name": "",
        "version": "",
        "repository":"",
        "licenses": ""
}

The output of the license-checker tool looks like:

Example — gradle based project

Gradle is a popular build tool currently. It allows to configure your dependencies in a declarative way (like maven). The dependencies look like:

Currently there are not many options to generate a list of licenses.
One option is the Gradle License Plugin

To use it, you have to configure it in your main gradle buildfile like this:

apply plugin: 'java'
apply plugin: "com.jaredsburrows.license"

licenseReport {
  generateHtmlReport = false
  generateJsonReport = true
// These options are ignored for Java projects
  copyHtmlReportToAssets = false
  copyJsonReportToAssets = false
}

Afterwards you can generate a json report it by using following command:

gradle licenseReport

The task generates also a json file besides the html — the output looks like:

This json file can be further processed to generate a csv file by using json2csv, a npm based tool.

The following command can transform the json output from the gradle plugin to a csv-file

json2csv -i licenseReport.json -p -u licenses -f project,version,url,licenses.license_url,licenses.license -d “|” -o licenseReport.csv

The csv which is generated looks like this in Excel (after seperating data via | symbol):

Summary

Two examples for npm and gradle based projects have shown that this boring task can be scripted and could be part of a continuous delivery pipeline, so that these reports are automatically created with each version of your software and so kept up to date automatically.

Nevertheless the quality of these reports are depending on the libraries being used because libraries have to maintain the metadata in their own projects. If your customer or you need a complete list, you have to manually edit the generated list. But the output is a good start and maybe sometimes enough as documentation.