Improving the Developer Experience — Our Application Security Journey (Part 3)
This is the third in a series of articles on the state of Application Security at Wise, describing our integration of security in the Software Development Lifecycle.
In part three of our blog post series we will be focusing on how we can improve the developers’ experience around fixing vulnerabilities. We explore how we identified the main pain points developers were facing, what we did to improve this experience and future improvement ideas that we are planning to work on.
Recap: What happened so far?
Over the course of the last year we have created our new setup for identifying vulnerabilities at Wise. It’s centred around DefectDojo as our vulnerability management tool and we use scanners, such as Trivy for Software Composition Analysis, as inputs. You can learn more about our setup in part one of this blog series.
Since then we have also created various dashboards to report vulnerabilities to our stakeholders, and also improved our Service-level agreement (SLA) with our developers to resolve new vulnerabilities. To learn more about that, please read part two of this series.
What issues are our developers facing?
As a next step, it’s important to check in with our developers to see how they’re interacting with our program. It’s one thing to set up a vulnerability management program, but we also need to make sure that it actually works for our users and that the recommendations provided are actioned in a timely manner.
While the initial feedback we received from developers was promising, we decided to share a survey with them to collect more actionable feedback. Our survey had a mix of multiple-choice and free-form questions and was mostly focused on how developers are using our tools and what issues they might be facing with them. We also interviewed them for analogous use cases, including what works well and what doesn’t work well in other tools they’re using. For example, we asked:
- When did you last check the status of your team’s vulnerabilities?
- If you already receive alerts, how do you receive the majority of them?
- How often would you want to be notified about vulnerabilities that were found?
Our survey concluded that there were two main pain points that developers were experiencing: Our dashboards were slow to load, and second, that teams had to be proactive and needed to remember to check the dashboards we provided them with, sometimes clashing with their already packed schedules and deadlines.
These two issues raised could potentially result in developers getting easily frustrated by our tooling, but also that they might forget or miss vulnerabilities, because they didn’t check a dashboard in time. The developer experience suffered from that, but it also made it harder for teams to stay within the SLAs that we established.
What can we do about it?
One simple way to tackle both of these issues at the same time is to introduce alerts. It solves both the speed issue, as developers don’t even need to visit the dashboards, unless we tell them there is something worth seeing there, but also the proactivity problem: Instead of developers constantly needing to check whether there are new vulnerabilities the alerts can just tell them.
More than half of the survey participants requested to get a weekly report of all vulnerabilities that are approaching their SLA due date, so that they would have enough time left to fix them. In general, team’s ways of working differ widely across the company though, as our teams have a lot of autonomy. To address that, we added a version-controlled configuration file that teams can append a cron expression to in order to receive notifications on their own schedule.
[
{
"teamName": "application-security",
"slackChannel": "appsec-alerts",
"cronSchedule": "0 9 * * MON"
},
// ...
]Code Snippet: Teams define their team name, preferred slack channel as well as a cron syntax schedule to receive alerts. In the example above, the Application Security team would receive their notifications at 09:00 am on Monday.
Almost all of our alerting in the company is delivered via Slack, which is why we decided to create a Slack notification as well. Our new bot now sends teams an automatic vulnerability report whenever they need it, which for most teams means once a week. The report shows all vulnerabilities that will breach their SLA in the next two weeks or less. For each vulnerability, the report also shows some metadata, including how many days are remaining to fix it, what the mitigation is and what the CVE of the vulnerability is.
Additionally, teams can report vulnerabilities as a false positive, directly from the slack message, by raising a pre-filled Jira ticket with us. One month after releasing this new feature more than half of all teams in Wise had already signed up to receive these notifications which is a great indicator.
What’s next?
Currently, the bot still has a few limitations that we are working on improving. Since releasing the bot we have also massively improved the loading speed of our vulnerability dashboards. Over the next months we will continue to improve our service to developers based on the feedback we receive.
We are also really excited about the developer portal that was recently launched in Wise and is discussed in more detail in our recent wise tech stack blog post. One aspect that’s particularly interesting about it is to display vulnerability data in the developer portal, similar to what Spotify recently described in their post about vulnerability management.
If you enjoyed reading this post and like the presented challenges, keep an eye out for open Application Security Engineering roles here.
Disclaimer: Product names mentioned in this article are property of their respective owners. This article is not aimed at publicising or endorsing any of the brands used or mentioned in the article.
