Penetration testing: Thinking in scenarios
Written by: Kevin Dunn

tl;dr — We explore what penetration testing’s various definitions are today and how scenario-based penetration testing allows organizations to gain further value by answering real-world threat scenario questions.
Penetration testing & its many meanings
Today, penetration testing is an overloaded term meaning various things to various organizations. When organizations discuss penetration tests, they often think about them in a number of ways depending on their familiarity, experiences and organizational cyber security maturity. In this blog post, we will explore the most common ways tests are thought about today.
Vulnerability assessments
Vulnerability assessments identify if technology is vulnerable to known weaknesses and/or if an organization is compliant (e.g. PCI etc.), typically in a highly automated fashion.
The goal? Being able to identify as many vulnerabilities as possible in the least intrusive manner.
Penetration testing
Penetration testing assesses the risk of compromise for a specific environment, system or application using all known techniques but often without wider context.
The goal? Being able to identify and demonstrate vulnerabilities within the scope of an application, network or host while achieving good coverage within the client’s budget.
Red teaming
Red teaming assesses end-to-end modelling of real-world threat actor techniques, tactics and procedures (TTPs) by targeting people, processes and technology against a set of scenarios/objectives.
The goal? To identify and exploit the path of least resistance — or the actual attack path — that a threat actor would likely take i.e. threat emulation.
Red teaming also provides the benefit of assessing and providing a sparring partner for Blue Teams (defensive) as the engagements are generally stealthy and thus, unlike penetration tests, are intended to stay under the radar.
Threat intelligence-led red teaming
This type of red teaming introduces the modelling of specific actor groups and their associated techniques, tactics and procedures by targeting people, processes and technology against agreed set scenarios/objectives.
This type of activity is increasingly driven out of regulatory frameworks such as CBEST (BoE/PRA — UK), TIBER (DnB — Netherlands) and ICAST (HKMA — Hong Kong).
The hybrid: scenario-based penetration testing
We are seeing an increasing demand for a service type not captured in the above that sits between red teaming (a generally wide scope with scenarios) and penetration testing (a generally tight scope without scenarios).
Namely, clients are asking for scenario-led penetration tests. But what is the difference?
The key difference is the questions that are answered by the activities.
Questions a penetration test answers
With a penetration test, the goal is to identify vulnerabilities within the scope of a system or application, often across an agreed number of person days.
This answers the question: ‘How vulnerable is the application, host or network?’ However, the answer provides very little context as to real-world impact or likelihood when compared against, for example, the business’s critical processes.
Questions a scenario-based penetration test answers
Scenario-based penetration testing instead has its foundations in allowing organizations to validate threat scenarios. This approach answers questions such as: ‘Could someone forge an email into our automated stock ordering system from outside our organization?’ and ‘Would our defence team detect this type of attack?’
In order for the threat scenarios to be known, this approach relies on the fact that there has been a threat modelling exercise and there is an understanding of the system, its dependencies end-to-end, and the underlying business processes and the resulting potential impacts.
Maturity matters
Scenario-based penetration testing looks to answer real-world risk questions as opposed to identifying solely technical debt which then needs to be translated into business impact by the organization.
For organizations that are a point of maturity in their risk identification and management evolution, scenario-based penetration testing offers significant real-world insights. However, to ensure ongoing coverage when adopting it as a model, business-as-usual vulnerability discovery does need to be present.
That being said, we would encourage all organizations to consider this approach when considering their next penetration test.
Engaging NCC Group
Want to know more? Speak to your account manager, e-mail response@nccgroup.trust or call:
- United Kingdom and Rest of World: +44 (0)161 826 7589
- North America: +1–800–813–3523
- Netherlands and BENELUX: +31 (0)15 284 79 99
- Denmark: +45 7020 7525
- Australia: +61 (0)2 9552 4451
- Singapore: +65 6800 0950
- UAE & Middle East: +971 54 493 7877
Published date: 28 July 2017
Originally published at www.nccgroup.trust.


