Evaluating Mobile Security Products: Compromised Devices
As described in our overview post on evaluating mobile security technologies, we are releasing a blog series to bring the community up to a collective understanding of the mobile threat landscape, explain our current thinking on evaluating mobile security products, and solicit feedback on both mobile threats as well as evaluation methodologies for the domain.
This post explores the last of four mobile threat scenarios: compromised mobile devices. We believe that evaluating compromised mobile device detection capabilities of mobile security products has immense challenges, and that it is not practical at this time. However, we are interested in input from the community on potential paths forward.
We invite feedback from the community on all aspects of this research, whether better insights into the threats that need to be defended against, types of products to evaluate, or how to evaluate technologies against these threats. Please reach out to the ATT&CK Evaluations team (evals@mitre-engenuity.org) with any comments or questions you might have to help shape our future work in this space.
Compromised Device
Adversaries who successfully exploit device vulnerabilities can subvert the device’s security capabilities, including the ability to protect sensitive data. Mobile devices may run out-of-date operating system versions that are susceptible to exploitation using publicly known vulnerabilities. Older devices may be considered end-of-life and no longer have updates available. Mobile devices running an up-to-date operating system version may be susceptible to vulnerabilities that are not yet publicly known (i.e., zero-day vulnerabilities).
Zero-click exploits that compromise iOS devices, providing the ability to surveil the user and collect sensitive data, have been discovered (T1404). Android exploits have similarly been discovered. However, Apple and Google continue to strengthen the security architectures of their devices over time, and examples such as the ZERODIUM exploit acquisition program offering up to $2,500,000 for Android zero click exploits and $2,000,000 for iOS zero click exploits imply that exploitation is difficult. The high cost of zero-day exploits that work against fully patched devices implies that they may be reserved for sophisticated or high-budget adversaries and only used on high value targets, but even if so, unpatched devices susceptible to older exploits are certainly a concern.
In our pilot evaluation effort, we tested the capabilities of mobile threat defense (MTD) products to detect compromised devices. On iOS, we identified a public exploit, found a device running a vulnerable operating system version, ran the exploit, and tested whether the MTD product detected the device’s exploitation. On Android, we simulated device exploitation by unlocking the device bootloader on a Google Pixel 2 XL and installing Magisk, a patched boot image that deliberately places the device in a compromised state enabling privileged access. After placing the devices in a compromised state, we tested the ability of MTD products to detect use of post-compromise techniques by collecting and exfiltrating sensitive data stored on the devices.
We faced extreme difficulties crafting repeatable device exploitation tests that capture what would occur in real-world conditions. Due to those difficulties, we do not believe it is practical to perform these tests as part of formal product evaluations, but we are open to suggestions on potential paths forward. It was a challenge to identify public exploits, and on iOS, to obtain devices running a vulnerable operating system version. Apple’s security architecture prevents downgrading the operating system version, making it difficult to construct repeatable tests. On Android, our method of simulating an exploited device was not necessarily indicative of what would occur during real device exploitation. When an MTD product did detect device exploitation, it was not always clear whether the product was only detecting a specific artifact of the exploit that could be easily modified by a sophisticated adversary or was detecting something more fundamental about device exploitation that cannot be evaded. The ability to detect one exploit doesn’t necessarily extrapolate to an ability to detect other exploits.
Despite our belief that exploit-focused tests are not practical at this time, the below table represents a best attempt to outline a methodology for compromised device testing.
Defensive Opportunities
Android provides two built-in device attestation capabilities that provide indications of device state: SafetyNet attestations and keystore attestations. Apps (including MTD agents) can invoke these capabilities to obtain signed attestations of the device’s integrity. These attestations are generated in isolated areas of the device that are difficult for adversaries to tamper with. During our pilot effort in 2018, the Magisk tool was able to fool SafetyNet into incorrectly asserting that the device integrity was not compromised, but Magisk was not able to interfere with the keystore attestation. SafetyNet has apparently since been improved.
Android also provides the SecurityLog and NetworkEvent APIs that may be useful for detecting post-compromise behaviors by adversaries, but we have not yet encountered any examples of how they can be used. We’d appreciate input on potential use of these APIs. We’re not aware of similar attestation or event log capabilities in iOS.
Feedback Requested on Compromised Mobile Device Testing Feasibility
This post explores challenges with detecting compromised mobile devices, potential test cases, and defensive opportunities.
We’d appreciate your input. We do not think it is practical to test mobile security product capabilities to detect compromised mobile devices at this time, but we are open to suggestions. Do you have observations of other forms of mobile device compromise not covered in this post? Are there additional test cases we should consider? Are there other defensive opportunities that we should be aware of? Do you have examples of how Android’s SecurityLog, NetworkEvent, or other capabilities may be leveraged, and whether any attestation or auditing capabilities are available on iOS that could be used?
This concludes, for now, our series on evaluating mobile security product capabilities to address specific threats. We plan to follow up soon with next steps to work together as a community to develop a common understanding of threats and building a methodology for evaluating products. Please send any feedback or suggestions to evals@mitre-engenuity.org.
© 2021 MITRE Engenuity. Approved for Public Release. Document number AT0015
