Upgrading dependencies doesn’t have to suck

A painless, automated process for upgrading dependencies

link

In light of the recent log4j security vulnerability; I think it’s particularly pertinent to talk about the process of keeping dependencies up to date. In addition to the (obvious) security implications, keeping libraries near the most recently released version has other benefits as well, such as:

• allows you to take advantage of internal optimizations, improving performance
• allows you to use new features
• ensures you have bug fixes so that your software works as expected

does anyone really like keeping dependencies up to date? (link)

But let’s be honest, keeping dependencies up to date is a pain. Because of this, my first recommendation is to decrease the attack surface area of your application by using managed services (like lambda) and to minimize the number of libraries you include in your project. However, no matter how minimalist you are with your application design, almost every component will have at least one included library (your service has to do something, right?). Hence the process of keeping these up to date becomes a critical (but admittedly boring) task. The rest of this article summarizes a painless way of doing this in Java (with gradle), and the concepts can be extrapolated to other languages as well.

Implementation

Add this block at the root of your build.gradle to specify that you wish to activate dependency locking

dependencyLocking {
   lockAllConfigurations()
}

you can specify this for just one configuration (such as implementation) if you so choose

Change how you declare the versions of your dependencies to use dynamic versioning (using wildcards) rather than hard-coding the version in the build.gradle

that’s it, you’re done!

Running the upgrade process

To recalculate (and lock) the version(s) to include in this build, run the following command. This locks the versions for every subsequent build until this command is ran again

./gradlew dependencies — write-locks

This creates a gradle.lockfile (similar to package-lock.json in npm) which “locks” the versions of dependencies. These are the versions that will be used until the command is ran again, which ensures you have a determinstic build.

an example of the generated gradle.lockfile

Process

While this technical setup is cool, it is worthless unless it is executed regularly. Here’s the processes we have in place to ensure dependency upgrades don’t fall by the wayside:

• set up a nightly batch job to upgrade dependencies and create a pr from it (script included below)
• the “rotation” member on our team comes in every morning, and checks if there is a pr containing upgrades
• If there is a pr but the merge checks fail (usually from compilation errors) we add a story to address compilation issue either this week or next
• if the pr passes the merge checks, then rotation person merges dependency upgrades to mainline, and our pipeline deploys the change all the way to *production

git clone <repoNameHere>
./gradlew dependencies --write-locks
if [[ $(git status --porcelain) ]]; then
  git checkout -b "upgrade-dev-dependencies-$(date)"
  git add .
  git commit -m "auto upgrade dependencies"
  git push -u origin HEAD
  gh pr create --title "upgrade dependencies" --body "upgrade dependencies"
else
  echo dependencies are at newest version so no need for upgrades
fi

we have strong levels of testing, monitoring, alarming, canary deploys, and automatic rollbacks so we can just merge code to mainline and assume it works unless we get a slack message telling us it didn’t.

Bonus Thoughts

Transitive Dependencies

The process above takes care of upgrading direct dependencies, but not transitive dependencies. Consider that you bring in apache commons, but apache commons has not yet cut a release using the newest version of log4j. The old (and vulnerable) log4j dependency will still be included in your application. In order to force upgrades of transitive dependencies, I suggest adding the following lines to your build.gradle (in the configuration block)

this will force all transitive dependencies of log4j to 2.15.0 (the patched version)

Merge this code to mainline, but be sure to have proper functional testing, canary deploys, alarms, rollbacks, and recovery mechanisms (such as dlqs), as there is a chance that this can result in a NoClassDefFoundError (if one of the transitive dependencies uses a version of log4j that is incompatible with 2.15.0). I think it’s much better to have this trigger alarms and know that an error is occuring rather than failing silently and leaving your application open to compromise by continuing to use the vulnerable version. In this case, you can either remove dependency, ask the maintainers of the library to cut a new release (upgrading your code will be easy, since you already have a process to upgrade), or as a last resort (if you cannot remove the library and an update is not immediately available), you can try to filter the incoming traffic using a WAF rule.

Multi-module builds

Unlike most tasks, running ./gradlew dependencies — write-locks from the parent project will NOT run the task of the same name in the subprojects. To do this, you can include this in your top-level build.gradle

subprojects {
    dependencyLocking {
        lockAllConfigurations()
    }
}

task upgradeDependencies(type: Exec) {
    List<Object> commandPartsList = new ArrayList<>()
    commandPartsList.add("./gradlew")
    getRootProject().getSubprojects().stream()
                                .map(Project::getName)
                                .forEach(projectName -> {
                                        commandPartsList.add(String.format("%s:dependencies", projectName))
                                        commandPartsList.add("--write-locks")
                                })
    Object[] commandParts = commandPartsList.toArray(new Object[commandPartsList.size()])
    commandLine commandParts
}

and then you can run this command from the root of your project to update dependencies in all subprojects (make sure to change the pr script to run this command as well). With this approach, no changes are needed in sub-modules to automatically upgrade dependencies.

./gradlew upgradeDependencies

Conclusion

This article has shown us that while upgrading libraries is boring, it doesn’t have to be painful. It details a simple way to automate this process (all it requires is adding three lines in your build.gradle and scheduling a gradle task to run) which ensures a strong security posture while allowing your developers focus on adding business value. We’ve also added a bonus section on transitive dependency management, that shows how to mitigate these risks while you wait for other teams to cut releases of their libraries you include in your project. Finally, we showed how to minimize the code duplication by registering a custom task to run this across all subprojects.