Threat modelling — a systemic approach
Introduction
Threat modelling is one of the most critical security-related activities we perform when we are building a new system or updating an existing one. Threat modelling allows us to find vulnerabilities, identify threats and attack vectors, associate and assess risks, and introduce controls to avoid, mitigate or transfer them.
Enterprise applications are incredibly complex, and performing threat modelling for such complex systems is a very tricky and complicated task. I, in my previous post, illustrated that security is a systemic concern. Therefore, analytical approaches that usually come to our rescue in managing complexity by breaking down a phenomenon into smaller, more understandable components cannot and must not be used for threat modelling purposes.
It is important to remember that as a systemic (type II) property, security is not transitive. This fact means, a collection of secure components [sub-systems], does not necessarily build a secure system (just like a collection of performant components does not necessarily produce a performant system).
Over the next few sections, I will introduce a holistic, systemic approach to threat modelling. This methodology keeps the essential properties of the system whilst managing its complexity. I will also explain how this methodology can be used in conjunction with agile practices.
Background
If you remember from the previous posts, synthesis is the appropriate method of enquiry (tool) for understanding systemic properties. Unlike analysis, which focuses on breaking the system down into smaller components to understand it, synthesis focuses on the environment and the system’s interaction with the environment to achieve the same objective.
Synthesis tries to manage the complexity of a system by taking advantage of the following concepts and thinking tools:
System Boundary: separates the system under investigation from the environment and other systems with which it interacts. Defining the boundary correctly is very important as it is the first step in understanding a system and determines the phenomenon on which we must focus.
Abstraction levels: Instead of breaking the system down, we look at it from different abstraction levels. This concept means the outer system is an environment in which small sub-systems interact. As we go up and down the abstraction levels, we still see systems and their environments — and their interactions with the environment, just like a conceptual fractal.
Perspectives: the point of view or the angle we look at a system. It allows us to focus our attention on the concepts included in that perspective. We need to bear in mind that perspectives are virtual concepts and are mainly used for categorisation and sub-categorisation. For example, we might be looking at a problem from the security perspective and then within security from an attacker point of view.
The next section will combine these concepts with the agile “little and often approach” to produce a systemic and agile threat modelling methodology.
The Methodology
This methodology revolves around the following main constructs:
The scope (the boundary)
Defining the scope of a threat modelling exercise is always a difficult and often contentious task. In this methodology, we use user stories as the basis of defining this scope and our system boundary.
In agile methodologies, user stories are informal descriptions of self-contained pieces of functionality, usually from an end user’s perspective. The system implementation is a piecemeal realisation of the user stories. In each iteration, we choose a number of user stories and build a system that realises the user stories’ objectives. Several components (sub-systems) interact to achieve the objective of one or more user stories.
The user stories selected to be implemented in one iteration are a good source of information for the threat modelling session's scope, or the system boundary under investigation in that session.
The components implemented (or designed, or planned to be implemented) in that iteration are considered in scope for the threat modelling exercise. Each system that contributes to the realisation of the user stories within this iteration is also in scope
This means the threat modelling exercise scope will be limited to newly developed components and the existing systems with which they interact to achieve the user stories' objectives [the system environment] and their integration points.
It is essential to remember that the users (who can also be the abusers) of the system are considered in scope as part of the socio-technical system. They integrate and interact with the system through the input they provide. It is also important to remember that communication channels are also considered a separate component that needs to be investigated.
The abstraction levels
We always start from the highest abstraction level when we perform the threat modelling exercise. It means we consider the components that have been developed and the existing system with which they have integrated as black boxes that interact with each other through data communication. This is why data flow diagrams are the chosen modelling tool for this methodology, but we will discuss this later.
When we are finished identifying the vulnerabilities and suggesting controls at that level, we choose a sub-system (we usually start from the newly developed ones) and delve deeper. It is important to highlight that we are not analysing each component in isolation. We are looking at each component as a white box which its own components can interact with the environment.
This process is repeated for all components and sub-components. If the immediate environment for a component has not changed, we do not need to delve deeper (try the lower levels of abstraction) for that component.
The perspectives
We can perform the threat modelling from several different perspectives at each abstraction level using different available tools and techniques associated with that specific perspective.
For instance, we can perform the threat modelling from the data security point of view using the STRIDE model, or we can perform the threat modelling from the attackers perspective using MITRE ATT&CK framework or by writing abuse cases or attack trees. By looking at the problem from different perspectives, we can improve our threat modelling exercise's completeness.
The system modelling tools
All modelling tools and techniques that can clarify the aforementioned points and help us manage the work's complexity are acceptable.
My experience shows that system component diagrams and data flow diagrams are the most useful diagrams in our modelling toolbox. The component model helps illustrate the systems at the right abstraction level, and the data flow diagrams show the flow of data, the lifeblood of our systems and components.
The process
Now that the main constructs have been introduced, we can explain how we can put these constructs together to complete our threat modelling exercise:
- Define the scope by looking at the user stories that have been chosen to be developed (or have been developed) in this iteration.
- Collaboratively create a system component model of the main components that realise the selected user stories' objectives. Make sure you add all users to the system model, and model the communication channels as separate components.
- Create a data flow diagram that clearly illustrates how data is being passed between different systems to achieve the scenario's objective for every scenario (main and exception) in the selected user stories.
- Using the system model and data flow diagrams, identify vulnerabilities looking at the problem from different perspectives (you can use STRIDE or Attack trees or other techniques in this step to focus your efforts).
- Look at each component in the system model. Identify the components that have recently been developed or the components which their immediate environments have significantly changed. Create a system model for the sub-components of that component and repeat the process from step 3.
- Continue until no component needs to be investigated from a lower abstraction level.
This process will be repeated at every iteration when new user stories are selected. Therefore the holistic threat model will evolve as we build the system iteratively.
Conclusion
In this article, I introduced a methodology according to which an iterative and evolutionary threat model can be produced based on systems concepts and considering security an essential property of the system under investigation.
We are combining different tools and techniques to achieve our objective, and creative holism which uses different systemic paradigms and looks at the problem from different perspectives in different abstraction levels is the basis of our approach.
In future posts, I will try to clarify the approach and the methodology by sharing an example of applying it.
