Shift-Left Cybersecurity at NS

Cybersecurity is a topic that cannot be ignored in today’s IT world. Systems are more connected than ever and therefore vulnerable to hacks and other cyber-threats. Cybersecurity is certainly not new within NS. We work hard to keep our systems secure. We set the bar high and are driven by an ambition to integrate cybersecurity further into the development process. After all, cybersecurity starts with developing secure software. In this article we take you into the world of shift-left cybersecurity. We explain what it is, how to apply it and how to get better at it.

Within NS, DevOps teams have been working on cybersecurity for some time. Making threat models, where security threats and risks are systematically mapped for applications, is part of our daily practice. Of course, we do this as part of the iterative refinement process. This keeps the scope manageable and the threat models always up to date. Of course, the threat model is tested during a penetration test and, in addition, the Red Team also periodically tests cybersecurity in practice.

Cybersecurity and DevOps

Most teams are already knowledgeable about cybersecurity. They are very aware of it during development, they do threat modeling, regularly have pen tests performed and sometimes even take some first steps with security-specific test automation. Nevertheless, there is often the feeling that a team can do more, especially given the daily DevOps practice in which they are used to testing as often and as early as possible. Still, doing a penetration test is often an after-the-fact activity when the software has already been delivered. Any findings from a penetration test are then often already beyond the focus of the team that is already fully engaged in the next sprint.

For functional tests, a familiar practice for teams is to move them forward in the development process, this is called shift-left. By purposefully using test automation, developing software goes hand-in-hand with the automation of test cases. This allows findings to be resolved during software development rather than after a release. This enables teams to deliver software faster by getting early feedback on functionality. Performing test automation within a CI/CD pipeline has thus become a crucial part of the DevOps way of working.

Is there also a way to get the same early feedback on cybersecurity?

Shift-left also for security

Fortunately, it is becoming increasingly easy to integrate all kinds of security testing tools into a CI/CD pipeline. The tools can be launched after a commit (or a periodic build such as a nightly) and, like their functional counterparts, provide comprehensive reports. Along with that comes an important challenge: A team that naively incorporates these tools within the pipeline will be overwhelmed by a mountain of findings. These findings may or may not be justified, false positives are often complex in terms of security with a lot of ifs and buts. Also, not every team has the proper knowledge to correctly assess CVSS (Common Vulnerability Scoring System, a standardized way to estimate security findings) and CVEs (Common Vulnerabilities and Exposures, publicly known vulnerabilities in software).

The above challenges often arise when a team starts working with security tools in an unstructured way. To help with this, we can distinguish a number of categories of security tools. By working with these categories in a focused manner, a team can grow in improving cybersecurity within the delivery process. The categories are as follows:

• SAST Tooling (Static Application Security Testing):
  These tools scan application source code for insecure programming constructs and other potential vulnerabilities. This is a white-box method of testing. Often teams are already familiar with this for functional purposes (e.g., determining code coverage metrics by SonarQube).
• DAST Tooling (Dynamic Application Security Testing):
  Once an application is compiled, this type of tooling dynamically tests it for vulnerabilities. An example is testing for the presence of the correct HTTP headers to prevent abuse by malicious actors. This type of testing is a black box approach.
• Secret Detection:
  To prevent “secrets” from being unintentionally revealed, this category of tools can be used. Consider passwords or certificates present in the codebase or revealed via a commit message.
• Dependency Scanning:
  It is important to understand an application’s dependencies (the libraries the application uses). Recent vulnerabilities such as those in Log4j emphasize this importance. By including tools from this category by default within a pipeline, there is always an up-to-date overview and it is possible to act quickly when a new CVE occurs.

Some tools in the above categories are easily deployed by a team. Consider scanning repositories for secrets or installing a linter that provides on-the-fly feedback on potentially insecure programming constructs. Other tools require some cybersecurity knowledge, such as configuring a DAST tool. To discover how best to help teams with a shift-left cybersecurity approach, we recently tried some of these tools in a hackathon.

Hacking at the Hackathon

The hackathon took place within one of our departments. During an afternoon several teams went to work with a selection of shift-left security tools. By testing these tools in a hackathon, we could immediately test their applicability to a real codebase and multiple applications. The teams consisted of a mix of security experts (pentesters) and team members from the development teams. In this way, each team had an optimal mix of cybersecurity and application knowledge. Moreover, this way of working together made for a very fun afternoon!

The results of the hackathon were surprising as well as practical for the participating development teams. For example, some tested linters turned out to add immediate value during the development process. The deployment of SAST tools also turned out to be easy to do without much in-depth security knowledge. The biggest surprises came mainly from testing dependency scanners. “Are we really still using all these?” and “This should be a standard pipeline for every team!” were just a few of the reactions.

So is it really that easy to include all security tools within a development pipeline? Unfortunately not, during the hackathon DAST tools turned out to be a bit more complicated to deploy. To arrive at usable reports, it is really necessary to configure these tools properly. One of the applications tested had a slight deviation in the standard HTTP responses; the tools could not handle this properly out-of-the-box. Also, the run times of dynamic scans were on the high side (tens of minutes), making them more suitable for nightly runs than standard inclusion in the build pipeline.

Next-Steps and Challenges

The hackathon taught us that it is not trivial for development teams to get started with shift-left security. While the tools are mature and comprehensive, deploying them effectively still requires some specific security knowledge. The latter is especially the case with DAST tools. SAST tools, dependency scanners and secret detection tools are closer to the development process. This makes them more suitable for teams to independently integrate into the pipeline. This brings us immediately to the first challenge.

Which pipeline?

Is it wise to include all security tools in teams’ own build pipeline? After the hackathon, we can answer this question with a “no.” Especially the tools that require the necessary security knowledge or have a long lead time you do not want to include indiscriminately within a buildpipeline. The longer lead time can possibly be solved by setting up a nightly build. However, you do want to avoid flooding the teams with all kinds of complicated and specific security findings. After all, all findings must be clear enough for a team to act on them immediately.

An alternative may be to create a generic security pipeline. This pipeline can even be organization-wide. In this way, findings per application can be bundled and (possibly supported by a baseline) fed back purposefully. This prevents a flood of false positives and allows teams to do what they do best; namely, develop and maintain software.

Clear findings?

As a developer, how do you deal with a CVE on a widely used library that also has a mountain of configuration options and is accessed via a web application firewall? This is a difficult question and not one that every team can answer immediately. Properly assessing the actual impact of a CVE, or any other security finding for that matter, requires specialized knowledge. That’s why it offers added value to group findings centrally and to call in the help of a security tester when things are unclear.

The shift-left security approach using a generic security pipeline offers opportunities in this regard. Security testers can help configure the various tools and also assess the findings generated. By setting this up smartly, you can also ensure that teams can learn from each other. After all, similar findings may have already been resolved by another team. As such, it is an effective way to foster mutual connection between teams.

All in all, the hackathon taught us that today’s security tools provide opportunities for teams. By deploying the right tools, they are able to actively engage in cybersecurity themselves during the development phase. Early feedback in this area is indispensable. Findings remain a focus, but with some help from security specialists, teams can grow in this as well. A shift-left cybersecurity approach is the way for teams to become and remain resilient!