Announcing Providence: Rapid Vulnerability Prevention
Security at high speed isn’t easy. New code is being written and deployed every day, and it’s hard to find the time to give all of it a thorough security review. Identifying patterns (and anti-patterns) that relate to security is a great approach, because it lends itself well to automation. There are many static and dynamic analysis solutions out there, of course, but we’ve found that many of them can be extremely resource-intensive or time-consuming. We were looking for a way to quickly address the set of issues that are relatively easy to find as patterns in source code.
That’s the reason we built Providence: a simple tool for commit-time source code analysis. Our goal was to prevent obvious bugs from ever being deployed into production in the first place, without spending all our time staring at source code. It’s already been a big success for us: the system has been been running well for the past year, and has found and prevented a number of bugs before they were able to reach production.
That’s why I am extremely excited to announce that we have open-sourced Providence; you can get the code here. The rest of this post will explain a bit about how it works, and what it has done for us.
What Is Providence?
Providence is a lightweight and scalable tool that finds bugs and anti-patterns in code commits. We’ve used it to prevent vulnerabilities ranging from XSS, to access control issues, to XXE. It works by continuously monitoring and pulling commits from version control systems and scanning them for bugs with rules defined in plugins. Additional plugins are easy to create and deploy, which has allowed for quick reaction to new bugs or problems as they are discovered.
Providence is easily integrated with SDLC workflows or bug-tracking tools. This model of addressing issues provides relative immediacy of resolution. On average, potential problems found by Providence are resolved more quickly than other vulnerabilities because developers are presented the issues immediately after they commit the code, instead of weeks to months later.
Benefits
Providence provides several benefits:
- It applies plugins to every line of every commit, providing complete coverage across your code.
- Rules are meant to be simple, e.g., matching a regex on a particular line of code.
- The immediacy of the results: issues are found at commit time, not days or weeks later when someone does a periodic codebase scan. When there is something to fix, the developer doesn’t have to context switch.
- Rules can be rapidly deployed. For example, if a new CVE comes out, a plugin can be quickly written and integrated the same day.
- Providence is lightweight and scalable. New plugins are easily deployed, and the failure of one plugin does not affect the others.
Providence Architecture
Providence works by polling one or more version control systems, and then running plugins against the commits. A plugin contains logic which analyzes the commit, looking for something suspicious. If something suspicious (determined by the logic of the plugin) is found, more operations can be performed. This can range from email alerts, to creating issues in an issue tracking system. To provide a more concrete example:
Let’s say you are concerned about the use of a system call in your Java codebase, performed via “Runtime.getRuntime().exec(<command>)”. You can write a plugin containing a regex to match that pattern — Runtime\.getRuntime\(\)\.exec\(, and logic to create a GitHub issue and email the security team. If a developer commits code into GitHub which contains this system call, the plugin’s regex will match, create the GitHub issue, and email the security team alerting them of the concern.
Plugins can examine diffs, full code, or whatever is necessary to determine if there is an issue.
At a lower level, Providence is composed of several components:
Scheduler
The scheduler kicks off the subprocesses of Providence at regular intervals, which is necessary when the VCS API doesn’t provide push capabilities (e.g., Perforce).
Watcher Engine
Watchers correspond to a particular VCS. They contain the code to interact with that API. These interactions range from monitoring commits to pulling down complete files for further analysis.
Rule Engine
The rule engine runs all the plugins you have defined. If a change meets the criteria of a plugin, the plugin’s rules are run. The criteria could be as simple as a regex, or more complicated (containing arbitrary logic). Alerts, logging, issues, and investigations can be created as well.
Plugins
Plugins contain the logic applied to the diff (or the file itself- whichever is fed into the plugin). They are the “magic” behind Providence. Plugins perform whatever analysis of a file is desired, and based on that analysis can create alerts, send emails, or perform operations in the SDLC workflow (via a custom integration).
Our Providence Implementation
We’ve deployed Providence in house by connecting it to both Perforce and our internal GitHub, and have been running it successfully for nearly a year. We expanded the number of plugins in use gradually, to ensure that Providence was never an obstacle for developers. We have integrated Providence with our internal issue-tracking system, and whenever a plugin finds an issue, it creates an investigation.
We have a snapshot of internal metrics from March to mid-August of 2015. During that time, Providence scanned 48,000 commits, fired 250 alerts, and led to prevention of 40 bugs. We currently have plugins which detect:
- use of runtime execution
- direct usage of crypto functions
- IP whitelisting bypass
- XXE
- secrets (like passwords) stored in the source code
- direct access to sensitive services (e.g., Hardware Security Modules) and
- XSS in HTML pages
Among many others. We are always compiling updated metrics, which we hope to share again in a future blog post.
The Future of Providence
We’re continuing to build new plugins and features, and we’re excited see what ideas others may have in mind! If you’re interested in our Providence talk from OWASP AppSec USA, you can find our slides here.
Please clone our repo, try it out, and tell us how it works for you! It’s on GitHub here.
