From spreadsheets to vulnerability management

Aug 3, 2020 · 5 min read

To identify security vulnerabilities more quickly the cyber security team at the FT is exploring building automation around security testing. In addition to this, we have started to implement a vulnerability management process by which we can automatically track these vulnerabilities and assign owners to findings.

Vulnerability management is the process of identifying, classifying, prioritising and remediating vulnerabilities in a continuous manner. But just what is the correct way to implement this in your organisation? There is no simple answer. It’s all about doing your research into what works best for the way teams in your organisation work and what will be most effective.

Keeping track of security findings

When security issues are identified, the first step is to track the issue in an agreed location. If you have ever worked in a security team in any organisation, you’ll know that keeping track of all the issues identified by automated scanners, bug bounty programs and penetration tests… can be a daunting task. The first problem that needs to be addressed is how and where do you track security findings? Maybe findings can all be tracked in a single spreadsheet?

While this may work for some organisations, having a single spreadsheet to track every security issue can get messy and very quickly get hard to use. Also, updating a spreadsheet for every security issue identified is a very manual process, and can often lead to single person dependencies.

Example of Capturing Vulnerabilities in a Spreadsheet
Example of Capturing Vulnerabilities in a Spreadsheet
Example of capturing vulnerabilities in a spreadsheet

Additionally, assigning owners to remediate the identified issues is a problem. Doing this through a spreadsheet doesn’t notify owners and leaves teams chasing individuals. Overall, the process isn’t smooth, it takes a great amount of time and spreadsheets are just not the best way to do this.

This is where we were, and it just wasn’t working. We needed some sort of system where interaction was seamless and could be automated. But just where to start!?

How to choose the correct product

Sure, there are various vulnerability management systems out there, but most are quite costly and not always customisable. At the FT, we opted for using OWASP’s open source project Defectdojo due to the product being largely customisable for our needs at the FT.

Defectdojo’s Main UI — Courtesy of Defectdojo
Defectdojo’s Main UI — Courtesy of Defectdojo
Defectdojo’s main UI — Courtesy of Defectdojo

Defectdojo allows you to maintain vulnerability findings in one centralised location and integrates with tooling such as Jira and Slack, used for tracking and notifying teams. It also provides support for the integration from multiple sources such as Nessus, Nmap and Hackerone, to name a few. Additionally, there are many options available to customise Defectdojo, thanks to its API.

Linking vulnerabilities to products

Having the ability to import findings into Defectdojo is great, but we also needed a way to assign the findings to products and product owners. At the FT, we use a custom built product called Biz Ops. Biz Ops is where the FT retains information about its business operations, such as what our systems are, who owns the systems and in-depth data about other functionality that may be useful to teams. All of it queryable via an API.

Each system or product within Biz Ops is linked with a corresponding “system code”. By utilising the Biz Ops API it was possible for us to integrate system codes into the “products” section of Defectdojo, thus having a way to link vulnerabilities to systems. Once you have a system code, there is a very high chance that you will also have a linked system owner.

Importing Products into Defectdojo
Importing Products into Defectdojo
Importing products into defectdojo

Now, with the ground work complete, we started to look at how to automate the process of populating findings directly from our current scanners into Defectdojo.

Importing all the things

As mentioned at the start of this post, Defectdojo provides support for uploading scan results from a variety of different tooling, including custom built scanners. By making use of the Defectdojo API it was possible for us to upload a variety of different report types generated by tooling used throughout the FT.

Our scanners all run on a scheduled basis and write their results to a common S3 bucket.

High level diagram of how the flow is automated
High level diagram of how the flow is automated
High level diagram of how the flow is automated

Going one step further to automate our vulnerability management process, we created Lambdas that would trigger every time a new object was put into the S3 bucket. Every time the Lambdas are triggered, they upload new scans to Defectdojo.

Additional tooling was created by the team to ensure that all scan names could be assigned to system codes before being written to S3. Having all scan results assigned to a system code and product owner inside Defectdojo means that there is no more trawling through outdated spreadsheets to find out who can fix serious security issues. Instead, the whole process of assigning results to products and products to owners is fully automated.

What’s up next!

Of course this isn’t perfect, we still have a long way to go. For now, we have set up regular scans that run on schedules, results that automatically export into S3 buckets on scan completion and finally, results that can be imported into our vulnerability management system which links to products and product owners, thanks to Biz Ops.

So, what’s next in our pipeline? Next up, we plan to surface the information contained in Defectdojo to teams by integrating with Jira and making further use of the Biz Ops API to ensure that tickets are created in the correct team’s Jira boards. This will bring us a step closer to automating the full end-to-end vulnerability management process.

FT Product & Technology

A blog by the Financial Times Product & Technology…

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store