Research Partnership explores Cloud Analytics
Written by Dr. Desiree Beck, Michael Butt, and Ingrid Skoog
Why Cloud Analytics?
While significant open-source cyber analytics exist for on-premises environments, the same does not exist for cloud platforms. As such, many defenders struggle to achieve similar levels of visibility in the cloud as they have on-prem. To address this problem, the Center for Threat-Informed Defense (Center), along with Citigroup Technology, Inc., CrowdStrike, Inc., Fujitsu, Google, LLC, HCA-Information Technology & Services, Inc., Microsoft Corporation, Siemens AG, Splunk, Inc., and Verizon Business Services launched the Cloud Analytics project, which researched and developed best practices to help defenders improve their ability to detect adversary behaviors in today’s complex cloud environments.
As JFK Jr. famously said in his 1962 moon speech, the Center chooses to do projects such as this, “not because they are easy, but because they are hard.” We began this research expecting challenges, and our expectations proved accurate — developing cloud analytics is a difficult task! In this post, we will discuss our approach and the complexities faced, as well as our end results — a set of cloud analytics for key tactics, techniques, and procedures (TTPs), as well as a “blueprint” document capturing best practices and lessons learned.
The output of this work is meant to serve as foundational knowledge upon which the community can build. Through further research and wider community involvement we will make further progress toward easier development of cloud analytics and expand the open-source corpus. If you have thoughts about the analytics or blueprint document, please let us know!
What was the approach?
The project used an iterative approach as shown in Figure 1. After standing up a test environment, we used the MITRE ATT&CK® Cloud Matrix as a reference for cloud-specific adversary behavior to identify a small set of cloud-native TTPs. We then emulated the TTPs in the cloud environment the developed analytics to identify the behavior. Cloud environments are very noisy, so developing strong analytics centers on reducing false positives.
The project focused on Azure and Google Cloud Platform (GCP) environments, but the results can be generalized to other cloud platforms, as well.
What analytics were identified?
The research team developed seven Azure and seven GCP analytics as shown in Table 1. Further details are available in the GitHub repository.
What were the challenges and lessons learned?
The project posed many challenges and lessons were learned throughout the research. Highlights include:
- Setting up a cloud infrastructure for the purposes of research oriented analytic development within a regulated organization is non-trivial due to funding requirements and bureaucratic hurdles.
- For simplicity and repeatability, automated Infrastructure as Code tools should be used to define, provision, deploy, and maintain infrastructure.
- Initially, the team developed and executed an emulation plan for a given adversary and then tried to identify possible analytics by studying log events. This method, however, creates an overwhelming amount of data. By shifting to the targeted approach outlined above, the project was able to identify analytics.
- Analytic developers must balance the impact of false positives; analytics with low signal-to-noise ratios require more staff time to process and draw attention away from true threats.
- Cloud platforms, such as Microsoft Azure and GCP, have significant differences in terms of organizational structure and resource organization. However, high-level cloud concepts can usually be mapped to the specifics of each platform. For rapid and repeatable analytics in multi-cloud environment, applying standardized data schema during the data ingestion is recommended.
- Adversaries in the cloud tend to cross the boundaries of infrastructure dynamically in fast succession. Mapping the analytics efforts across infrastructure and cloud services can reduce one-off noises while increasing observability for abnormal and evasive behaviors quicker than analyzing each cloud infrastructure in isolation.
Where can I get further details?
The blueprint document contains details of the analytic development process and further expands upon the challenges and lessons learned shared above. It discusses an initial approach that didn’t work for the project, as well as potential problems with infrastructure setup. It also includes a case study that walks through the process of identifying a TTP, emulating the adversary, and developing the corresponding analytic.
The Cloud Analytics GitHub repository contains the Azure and GCP analytics, as well as the adversary emulation plans, emulation tips, and some additional support resources.
About the Center for Threat-Informed Defense
The Center is a non-profit, privately funded research and development organization operated by MITRE Engenuity. The Center’s mission is to advance the state of the art and the state of the practice in threat-informed defense globally. Comprised of participant organizations from around the globe with highly sophisticated security teams, the Center builds on MITRE ATT&CK®, an important foundation for threat-informed defense used by security teams and vendors in their enterprise security operations. Because the Center operates for the public good, outputs of its research and development are available publicly and for the benefit of all.
© 2022 MITRE Engenuity. Approved for Public Release. Document number CT0053.
