Three approaches to offensive security and how we choose between them
Mike Goodwin, Sage Technical Fellow and VP Product Security & Architecture
In most aspects of life, prevention is better than cure. In application security, this means finding potential security problems — known as vulnerabilities — before they are found by attackers. To do this we invest in security at all stages of the Software Development Lifecycle (SDLC) from security training for engineers, through design-time activities like application threat modeling and checking of code using Static Application Security Testing (SAST), amongst other things. All these activities are intended to catch vulnerabilities as early as possible — shifting security to the left as the industry jargon puts it — carried out by builders.
Despite all this effort though, vulnerabilities inevitably get through. We are only human after all. This means there will always be a need for breakers — attack-minded individuals, who look at the applications we are building to seek out vulnerabilities so we can fix them before real attackers can find and exploit them. This is known as offensive security.
For many years at Sage, we have routinely engaged specialist external offensive security companies for this purpose and for the most part, they have done a good job for us. They have found a whole range of vulnerabilities that could have left us, and our customers, exposed to attack if they had gone undiscovered.
Over time though, we saw the number of findings from these tests begin to fall away. We tried using different offensive security companies, but the result was the same. We concluded that all the effort we have been putting into securing our SDLC was working — there were fewer vulnerabilities making it through. Great!
However, rather than pat each other on the back and declare victory, we decided to evolve our offensive security practices. Today, we have arrived at a blended approach where, broadly speaking, we choose from three different forms of offensive security:
1. Application penetration testing
2. Bug bounty programs
3. Red team exercises
The way we choose between the different approaches in any given scenario is discussed later in this article.
Application Penetration Testing
This is what we at Sage consider our “traditional” approach to offensive security. They are characterised as follows:
· Testing carried out by contracted specialists or Sage colleagues, using a range of manual and tool-supported techniques
· Often targeted at common application vulnerabilities such as the OWASP Top 10
· Focussed on a fixed scope defined by a range of IP addresses, domains or URLs — at Sage, we almost always do penetration testing on non-production environments
· Restricted in some way to prevent unanticipated consequences or accidental damage — for example, we often ask the testers to avoid destructive testing such as DDoS attacks
· Delivered at a fixed cost over a specified number of days or weeks
This is our baseline approach for new products/services or where the SDLC maturity of the engineering team involved is relatively low.
Bug Bounty Programs
A relatively recent innovation compared to Application Penetration Testing, Bug Bounty Programs are a form of crowd-sourced offensive security. They are characterised as follows:
· Can be public — where anyone in the world can submit a vulnerability and get a bounty — or private — where the program is open only to an invited set of security researchers
· Testing is carried out by independent security researchers. For public programs, anyone is free to participate. For private programs, the researchers are invited from a pool and the typical number of participants is 10–50.
· With the right participants, Bug Bounty Programs can uncover a wide range of vulnerabilities compared to Application Penetration tests. They can also be constrained by a pre-defined policy if required.
· May be focussed on a fixed set of targets, but often carried out on production environments
· For each valid vulnerability they find, participants are paid a bounty, the size of which is based on the severity of the vulnerability
So far, Sage has not started a public program, but this is an aim for the future.
Red Teaming is a form of offensive security that is designed to closely simulate the behaviour of real attackers.
· Can be carried out by specialist contractors or internal teams
· Can use a wide range of attack methods including open-source intelligence gathering (OSINT), social engineering, physical intrusion testing, network reconnaissance and exploit kits
· For us, it is more loosely scoped than application penetration testing and bug bounty programs. In some cases, any asset or endpoint is a valid target. However, it can also be narrowly targeted around specific types of threat if needed.
· When carried out by third parties, it is usually on a fixed cost basis
Sage has established an internal Red Team made up of dedicated specialists, but we have also used external contractors where appropriate.
How we choose
Sage is large organisation with many teams working on a range of products and services at different stages in their lifecycle. To be effective and efficient at finding vulnerabilities, we need to choose the right offensive security approach at the right time.
We would tend to use an Application Penetration Test when:
· The product or service is new and has never been assessed before, because it is likely to result in a good number of findings at a reasonable cost.
· We want the test to follow a defined methodology, chosen by us
· The assessment calls for a particular technical specialisation that is hard to find, because we can select a supplier that can provide precisely the skills we need
· The product or service is complex to configure and use because we can offer the right level of preparation to the suppliers before the test starts.
We would lean towards a Bug Bounty Program when:
· The product or service is mature and previous tests have few findings, because then it is likely to uncover more subtle or obscure findings at a reasonable cost.
· The product or service is easy to sign-up to and use, because the low barrier to entry makes it more attractive to the bug bounty researchers.
Finally, we would consider Red Teaming when:
· We need to assess insider threats, because we need to engage in social engineering or similar methods.
· The system is complex, with many interacting moving parts, because the loose scoping of Red Teaming is well suited to these systems.
· We want to test the cumulative effect of multiple threats chained together
This is not an exact science though. Sometimes, other specific considerations come into play and common sense should always be the deciding factor.
Bonus Approach: Specialised offensive security projects
The three approaches outlined above have served us well in a lot of different situations, but there will occasionally be cases where a novel approach is needed. Following the well-publicised NotPetya ransomware outbreak in 2017, NCC Group were approached by a client (not Sage!) who asked the following question:
“The thing with NotPetya and our network is that we have made a number of assumptions about the defenses we have put in place and how these should have stopped NotPetya from spreading. But these are assumptions [and] I don’t really want to be dependent on them.
“So, would NCC Group be interested in producing a NotPetya simulation program? i.e. a NotPetya clone that we can run inside of our network, but with the ransomware removed and safeguards to ensure it stays within our network. Also, could you create some reporting so that we can understand what mechanism it used to move between each host and how long it took to move around the network?”
As a project, this is fascinating and potentially dangerous in equal amounts. Clearly it calls for a novel and innovative approach that is way beyond any of the approaches discussed earlier in this article — NCC took on the project and published the results as a great case study in a series of blog posts.