Security Testing — SAST, DAST and IAST explained
From our previous blog post Changes in OWASP Top 10: 2017 vs 2021 we know that there are many vulnerabilities covered by the OWASP Top 10. The bad news is, that these are so numerable that is impossible to detect all of them manually. The good news, on the other hand, is that there are also a lot of tools available for security testing. Starting from simple linters like spot-bugs to big security testing applications like the OWASP Zed Attack Proxy. To understand which tools we should use, we must understand the difference between them. It is possible to categorize them into three security testing categories:
- Static application security testing
- Dynamic application security testing
- Interactive application security testing
Each category has a special goal and features. But before we start digging deeper into each category, let’s talk about the connection between these categories and the Software Development Lifecycle (SDLC).
The Software Development Lifecycle (re-)visited
The SDLC covers everything from the initial planning and requirements elicitation to the long-term maintenance and eventual phasing out of the application. In the agile context, each change or feature is going through these phases and afterward the next iteration starts. If you look at these phases it is absolutely clear that a security issue found in production (Phase 6 — maintenance) is a bigger problem as an issue detected in the implementation phase. It is also clear that the later your security testing is positioned in your SDLC the later you will find vulnerabilities in your application.
But security testing is time-consuming and therefore expensive and often left out. But if you skip it completely, Hackers will do it for you in your production environment and that’s the most expensive and problematic way to find out that there are security problems in your application.
So, the key is to detect security problems as soon as possible. Many security blog posts call this the shift left in the development phase.
To create secure software (if this is even possible) we must integrate automated security testing as soon as possible. Some vulnerabilities are obvious in the code. They could be detected in the implementation phase. But that’s not a solution for all vulnerabilities. Some could only be found when the application is running in the later testing phase. For security testing in the implementation phase SAST tools are the right choice. For security testing against a running application DAST tools are right. Let’s start with SAST.
Static application security testing (SAST)
Static code analysis is a software review process that examines source code for quality, reliability, and security without executing the code.
This analysis can be used to identify bugs and security vulnerabilities that compromise the security of the application.
Static code analysis is fast and covers the whole code base. But the scanned code is not executed and therefore there is no runtime information available. Also, the code of third-party libraries and dependencies could not be scanned. Because of the ability to identify vulnerabilities directly in the code and the fast feedback they provide, SAST tools are normally used early in the implementation phase of the software development life cycle.
Many modern IDEs provide a SAST integration, and it is also common to use SAST tests in the build stage to the CI/CD pipeline before the application is deployed.
This enforces that all contributors adhere to strict development standards and the generated SAST test reports could be used as a quality gate, which helps to avoid that vulnerable code is getting integrated. With the coverage reports it is also easy to find out the amount of testing that has been done on an application.
Overall SAST’s brings many features. The most important are:
- Error detection
It can identify many bugs related to parallelism, corrupted data, data flow, security, and static and dynamic memory. Some of the bugs found are hard to detect with dynamic tests. - Low Cost
Static code analysis can be easily automated. There is also a lot of tooling available in the open source and commercial environment. - Compliance to coding standards
It checks whether the code complies with coding standards such as MISRA C or JSF++, with security standards such as CWE, CERT C/C++ and ISO/IEC 17961, or with cybersecurity guidelines.
Well-known tools are:
Dynamic application security testing (DAST)
In contrast to SAST, the scanning tools used for dynamic application security testing are developed to identify vulnerabilities during runtime. As a result, DAST tests are executed against a running application to detect potentially exploitable vulnerabilities.
These include, for example, configuration errors that would only occur in a realistic execution environment and that could not be found by just analyzing a piece of code without the execution context.
To achieve this, DAST tools are using a list of known vulnerabilities and malicious inputs to fuzz the application. Malicious inputs can be:
- SQL queries to identify SQL injections.
- Large inputs in strings to exploit buffer overflow vulnerabilities.
- Unexpected input data to detect invalid assumptions made by developers.
During the scan, the running application is attacked with these potentially malicious inputs. Afterwards the DAST tool analyses the application’s responses.
For example, if the application responds negatively or not as desired to the input or even crashes, the DAST tool records a vulnerability that may have been identified.
Because DAST tools are run against a running application, they can identify a wide range of potential vulnerabilities. They provide an easy way to detect vulnerabilities that are difficult or impossible to find in the source code. These include problems with memory allocation or insufficient input escaping.
As mentioned before DAST needs a running application with a working execution environment. Because of that, DAST typically comes into play in the testing phase of the SDLC. Once the application’s code can be built and deployed to a test or staging environment, you can use DAST and test it with simulated malicious input to detect vulnerabilities.
In contrast to SAST, DAST scans the runtime environment, not the source code. This brings many advantages:
- Vulnerabilities in dependencies
Because of the runtime scan, DAST also scans the execution of application dependencies. - Misconfiguration detection
As DAST scans a runtime environment, it is possible to detect misconfigurations that end in a security problem. One example for such a misconfiguration is: A4:2017-XML External Entities (XXE) - Finds vulnerabilities that are obvious
DAST uses well known attack vectors for its tests. Vulnerabilities that are detected by a DAST scan could easily be found by an attacker in production, too.
But DAST also has some disadvantages. Beside the need of a working runtime environment, is the amount of time that a full DAST scan needs to check the application. You can compare the time consumption of DAST tests with end-to-end system tests.
Well-known tools are:
Interactive application security testing (IAST)
Scanning the runtime behavior of an application brings many advantages. But in case of DAST the test results are independent from the code. In other words, DAST is a black box test. The test can only see the output or result of the application. SAST scanners on the other hand are whitebox tests. They have access to the complete code base, but they miss the runtime information.
IAST taggles this and upgrades the DAST black box into a white box runtime test. IAST scans have access to both runtime information and the codebase. They execute the code step by step; like a SAST scanner would do, but in the runtime environment. It could check what happens with the input and where the data is going. It scans the code (static) and the result (dynamic) at the same time and combines the possibilities of static and dynamic code analysis.
This significantly improves the test results as IAST knows the data flow and could reduce false positives as they really know if a passed input value was correctly validated and escaped or not. Unlike SAST scanners IAST also scans the code of dependencies and how they interact with each other. This brings the possibility to detect security problems triggered by a misused dependency or inside a library.
To analyze the runtime IAST scanners need an agent running along the tested application. This agent adds and injects sensors to the application that are gathering the relevant data. It works the same way as a web application firewall (WAF). You can imagine it like an automated debugger that steps through the code and runs a SAST scanner over the current line in the stack trace. If a problem or vulnerability is detected the agent reports it to the IAST scanner.
Unlike DAST, there is no need to write extra security tests. IAST scanners can analyze the data flow of the regular test base and report vulnerabilities, there is no need for special malicious input, as the IAST scanner could check the code if the input is escaped correctly.
This brings the possibility to just adapt the normal test stages of a CI/CD pipeline to run an IAST scanner along the normal test run. It is not necessary to create an extra pipeline stage or write extra tests to get a first IAST test report.
But like DAST this type of testing also does not test the whole application or code base. As IAST scans only the executed code of the functional tests, the code coverage of the test base directly corresponds to the coverage of the security tests. If problematic code is not covered by a test and therefore is not executed, an IAST scanner could not find any vulnerabilities in it. Introducing an IAST scanner to an application with a low test coverage brings nothing. So, code coverage is key here. It is therefore recommended to use IAST scanners also during development. This way security problems are detected while the developer runs some manual tests during development. Security problems detected in this early state of the software development life cycle can normally quickly be fixed. This is a huge advantage compared to DAST scans, which are placed much later in the QA phase. In this phase each detected vulnerability needs a bug report or change request.
To introduce IAST an agent for the specific language of the application is needed. There is no generic solution for this and there are not that many IAST scanners available right now. But IAST is a relatively new kind of security testing, maybe that will change in the future.
Currently the following tools support IAST:
Overview and conclusion
As you can see, SAST, DAST and IAST are different techniques to detect security problems in an application. Static code analysis is very fast and easy to integrate into the implementation phase. It tests the full application codebase and without the need of any extra tests. But because of the missing runtime information it can’t detect all vulnerabilities correctly.
DAST on the other hand scans the application without knowledge of the internal code. It just checks the output of the application for security vulnerabilities. For DAST you must write special security related tests. DAST test runs are normally integrated into the testing phase of the software development lifecycle. They are time consuming, but the results are good and important. So, it’s worth it.
IAST analyzes the data flow in the application and reports detected security problems while the code is executed. It can be easily integrated into a DAST scan to improve the outcome of the security tests. But it could also run while the execution of the existing functional test base. Also, developers could use IAST while manual testing during development. This will shift the runtime security testing even more left in the SDLC into the development phase, where problems could be more easily fixed without overhead. In comparison to SAST, IAST also scans the code of the application dependencies. But it only scans the code that is covered by the test base. Without a high test coverage, it brings no benefits. So, it is more an extension for DAST where security relevant parts are explicit tested in separate tests and not a replacement for it.
Thats it for now, Thanks for reading! In our next blog post we will analyze what of the OWASP Top 10 could be automatically tested using different tools and how we could integrate these tests and tools in a CI/CD pipeline.
If you have any questions, suggestions, or criticism on this topic, please feel free to contact me.
Special thanks also got to my colleague Heinz-Werner Haas.
You might be interested in the other posts published in the Digital Frontiers blog, announced on our Twitter account.
