Threat modeling Handbook #4: How to get better at Threat modeling
I know the previous story of this series (links to all previous stories below) was a bit long and may have been a little overwhelming. Hence, I have decided to dedicate this story to reflecting on the example we discussed in the previous story and share some tips on how to get better at threat identification which is the first phase of the threat modeling process as we have discussed earlier.
Threat modeling Handbook #1: What is a threat model and why your service needs one
Threat Modeling Handbook #2: Threat modeling is a process, not a document
Threat modeling Handbook #3: The Thought process behind identifying threats (with an Example)
The 3 domains of knowledge needed for Threat modeling
To be able to generate a useful threat model for a specific application or service, 3 different domains of knowledge are needed:
1- Technology: Understanding the technologies, languages, frameworks, libraries, platforms, .. etc used. e.g. For the example mentioned in the previous story, we need to have some understanding of:
- Python and Django (the backend programming framework)
- Javascript and React (the front-end framework)
- MySQL (the database)
- AWS services such as ALB, S3, EC2, RDS and Route53 (the platform / cloud provider)
- Other software running on the instance such as SSH.
2- Application: Understanding the application being reviewed and how it works. e.g. For the example mentioned in the previous story, we need to have some understanding of:
- What data is being handled and stored.
- The flow of this data and the places where it is being handled or stored.
- The different routes and entry points that can be used to interact with the service.
- The different kinds of credentials that could be used to access the data.
3- Security: Understanding the high-level risks and the different attacks, vulnerabilities, and misconfigurations that could lead to these high-level risks being possible. e.g. For the example we mentioned in the previous story, we needed to have some understanding of:
- The web application attacks (e.g. OWASP Top 10) that are relevant to the design (e.g. XSS and CSRF), their impact, and possible mitigations.
- The different kinds of vulnerabilities relevant to the application’s code (e.g. SQL injection), their impact, and possible mitigations.
- The different kinds of misconfigurations that were relevant to the design (e.g. Making the S3 bucket public by mistake), their impact, and possible mitigations.
Get the right people in the room
All the 3 domains of knowledge mentioned above are needed to be able to connect the dots and identify the possible threats. However, as you can imagine, in reality, it is very hard for one person to have the needed level of understanding and experience in all the 3 domains. That is why the first phase of threat modeling (threat identification) needs to be a group effort.
We need enough people in the room (or the Zoom meeting) to cover all the 3 domains of knowledge. For some projects that would be a developer that covers both the “Technology” and the “Application” and an application security engineer that covers the “Security” domain. For some other projects, we would need more people. For example, we may need involvement from:
- The platform team to cover the infrastructure configuration (part of the “Technology” domain).
- A senior or principal engineer who has more context about another service being used as a dependency (part of the “Application” domain).
- A senior or principal security engineer who has more context about a specific domain of attacks that may be relevant to your project (part of the “Security” domain).
That being said we should also try and make sure the process is as efficient as possible and be mindful of everyone’s time. Hence, my recommendation is to always start with one security engineer and at least one engineer who is involved in the design and development of the application, and pull in more people if needed to answer the specific questions we need (they don’t need to be involved in the whole process).
Have the right distribution of knowledge
Keep in mind that these 3 domains of knowledge are not distinct, in fact, they are quite related as they cover the same thing (Software) from different angles. That is why the more you learn about “Application” and “Technology” the more you know about the “Security” domain, and vice versa.
Hence, the above diagram showing the 3 domains of knowledge needed for threat modeling shouldn’t be misunderstood to imply that security engineers only need to have experience in the “Security” domain, and developers only need to have experience in the “Application” and “Technology” domains, as this would make the discussion and the collaboration quite hard and inefficient.
Instead, for the threat modeling discussion to be efficient each person involved in the process should have some level of experience in each of the domains that they are not primarily prominent in. That is why security engineers need to learn as much as they can about the code and the technologies used in their organization, and for developers to go through security training covering the main attacks relevant to the types of applications they are creating.
Here’s an example of 2 different distributions of knowledge in the 3 domains, and while both have coverage for all the domains, the first one makes the discussion very in-efficient, while the second one sets the discussion for success.
Tip#1: As a security engineer you can get better at threat modeling by learning more about the “Application” and “Technology” domains. Be comfortable around code, learn the frameworks, platforms used in organization, .. etc.
Lead the conversation
From my experience, trying to create threat models in fluid brainstorming sessions is usually wasteful and inaccurate. Instead, my recommendation is for the security engineer to lead the discussion in a way that efficiently consolidates all the context needed from the “Technology” and “Application” domains, and then apply this context to the “Security” domain experience he/she has to identify the threats.
Having a kickoff questionnaire (like the one mentioned in story 2, and used in story 3) is a good way to guide the discussion in this direction, but keep in mind that sometimes the questions in this questionnaire are not enough or irrelevant to a specific project, so we need to be flexible in these situations and adapt the questions to the project.
Thinking about attacks the “Threat modeling” way
Now with enough context about the “Technology” and the “Application”, here comes the hard part; Identifying the threats. This is the key skill we need for Threat modeling to work, and this comes down to knowing the attacks, vulnerabilities, misconfigurations, .. etc but from a very specific angle; the context they are applicable in.
If you look up any attack, you will find most resources discussing “How it works” and “How to prevent it”. This is very useful of course, but the question we need to answer first is “Is this relevant to our application”.
For example, if you look up CSRF (Cross-site request forgery) you will find a lot of resources on “How it works”, e.g. it can be exploited through sending a link to the victim via email. You will also find a lot of resources on “How to prevent it”, e.g. using a CSRF token. However, it is “What context it is relevant to” is not instantly obvious.
To answer the “What context is it relevant to” question, we need to understand the conditions that need to be met for the attack to be possible. In the case of CSRF, the goal of the attack is to trick the user into opening a link that does something mutating (e.g. share a file) from the victim’s browser so that it can use the credentials already in the browser without the victim noticing. From that, we can deduce the conditions are:
- A web application running in the user’s browser.
- The web application is using credentials kept in the browser to communicate with a backend API. (e.g. In story 3 we were using the session token which is stored as a cookie in the browser to communicate with the backend API).
- The backend API is used for mutating actions. (e.g. share a file)
If these conditions are met then CSRF is a relevant threat and needs to be added to our threat model, otherwise if any of the above conditions are not met (e.g. backend is ONLY used for read-only actions) then we can safely ignore it.
Let’s take SQL injection as another example, the conditions for it being relevant are:
- We have a service accepting user input directly (e.g. API input parameter) or indirectly (e.g. User input stored and used later for a scheduled workflow).
- This user input is used to build an SQL query and run it against an SQL database. (e.g. We take the file ID as input and use it to look up the owner of the file from the DB using a query like the one below, Remember the Source Sink model we discussed in story 3).
SELECT * FROM FILES WHERE FILE_ID = <INPUT_FILE_ID>;The “Threat modeling” decision tree
When I was learning Threat modeling one tool that helped me build the right mental model to be able to answer the question “What context is it relevant to?” for the different kinds of attacks was building a decision tree. This helped me visualize the conditions needed for each threat to be relevant (I used a mind-mapping tool for building the decision tree). And whenever I learned about a new type of attack, I would add it to the decision tree, this way I trained my mind to always think about the “What context is it relevant to?” question.
In this decision tree, I would keep the different kinds of components I typically came across in my reviews, the questions that map to the conditions, the attacks, and the suggested mitigations. Here’s a simplified version that has 5 different attacks.
As time passed this got bigger and bigger, but also I became less and less dependent on it as my brain became more trained on identifying the conditions needed for an attack to be relevant and applying them to the designs I was reviewing, and retrospectively I think that is the real benefit or keeping a decision tree; training my brain to see the right patterns.
NOTE: If you are interested in a dedicated story about the decision tree approach let me know in the comments. If enough people are interested I will work on it.
NOTE: This decision tree is also useful for automation, which will be covered in a future story in this series.
Tip#2: Keep a decision tree that shows the different conditions for an attack to be relevant, and make sure to maintain it, and keep adding any new attacks you come across to it.
Action plan for building the decision tree
The decision tree serves as the framework for us to learn about threat identification, however, it is only as useful as the number of relevant attacks included in it. Hence, we need to keep learning and growing the decision tree for it to be useful.
If you are new to threat modeling, here is a suggested action plan for you:
- Start with the basics: Learn about the OWASP Top 10 and CWE Top 25, and add them to your decision tree. Remember the focus is on the conditions under which they are relevant, so you don’t need to go very deep with all of them.
- Follow the news: Try to make it a habit to follow security news, bug bounty reports, vulnerability write-ups, .. etc. Whenever you meet a new story mentioning a new type of attack, read more about it, and add it to your decision tree.
- Create feedback loops: If you are working with security tools (e.g. SAST or DAST tools) or if you have access to pentest reports, check any valid findings that come out, and add them to the decision tree if possible.
This is a continuous and iterative process that takes time but it will help make the best of the decision tree. Also, it will help you train your mind to see the right patterns, and after a while, you may no longer be dependent on it as I mentioned earlier. In a sense, consider the decision tree to be like training wheels, they make starting easier, but as you get better you can do without them.
Suggested Exercise: For all the threats identified in the previous story, build a decision tree showing when they are relevant. Let me know in the comments if you need help going through this.
Conclusion
I hope after reading this story, you can now make more sense of the thought process followed in the previous story and also have a clearer pathway to building the skills needed for threat modeling.
Remember that for threat modeling to work you need to have the right context and for that you need the right people to be involved to cover all the 3 domains of knowledge needed “Technology”, “Application”, and “Security”, and to be able to use this context to identify the threats you needed to think about different attacks the “Threat modeling” way which you can train yourself on by following the action plan suggested above.
In the next story of this series, we will go through Phases 2 and 3 of the threat modeling process; Verification and Continuous testing. Stay tuned!
