Highlighting the Importance of Detection Context using the ATT&CK Evaluations for ICS Results
(Note: The content of this post was also released in a presentation format for the 2021 DEF CON ICS Village and can be accessed here.)
In the recently released results for the inaugural round of MITRE Engenuity ATT&CK Evaluations for Industrial Control Systems (ICS) we saw a high level of visibility accomplished by the participant platforms. The definition of visibility for this evaluation is the proportion of substeps with either an analytic or telemetry detection, accomplished by the participants. This is a great indicator of a platform’s capability to capture a large portion of the individual actions that were leveraged against the evaluation environment. Having this accurate record of information is critical to many investigative activities such as threat hunting, incident response, and forensics.
These activities, however, require human interaction. Given that it may be difficult for asset owners to hire and retain human resources with the necessary skills to effectively make sense of this voluminous data, we believe that tools should place greater emphasis on contextualizing the data. Context can provide greater awareness as to why a detection is relevant and how it relates to other events captured in the platform. This could be a force multiplier in the industry’s quest to effectively monitor Operational Technology (OT) networks. It should also be noted that while additional context can in many ways be the difference between detecting or not detecting adversary behavior, different behaviors will require different levels of context. Jamie Williams wrote a two-part blog series on this topic, that can also apply in many regards to the ICS domain (part 1; part 2).
In this post, we will highlight what we envision ICS detection platforms providing end-users in terms of detection context, the reality of where they are today based on our evaluation, and some key steps that can be taken to help fill what we see as a detection context gap.
Our Dream Versus Reality
In a perfect world, detection platforms would quickly and accurately identify all adversary behavior in your network. An analyst would not have to worry about false positives because the platform would have the ability to properly classify standard actions of legitimate users and the control system. A sleek user interface (UI) would highlight all noteworthy actions in a timeline view, linking together related actions and giving relevant information about how each action could affect the OT network, control systems and industrial processes. In addition, the platform would provide a coherent narrative of adversary actions while including supporting details in the form of telemetry.
While that may be what “perfect” looks like, we do not think it’s entirely attainable or should even be sought after. It is akin to asking for a perfect security posture. But elements of it are plausible and should be attempted by vendors in this space. The capability to have this type of awareness and convey it to the user was the vision of the research community before the ICS detection market existed. In addition to being able to parse OT specific data sources and alert on suspicious behavior, these platforms should provide a level of context comparable to that of a narrative as a defining capability.
In an effort to incentivize the addition of more context, we use detection categories, where general, tactic, and technique each indicate additional levels of granularity in terms of the amount of details provided to the analyst to aid them in understanding the behavior. In addition, to indicate that detections relate to each other and other data, we included the detection modifier, Correlated, which was defined as:
Data is presented as being descendant of events previously identified as suspicious/malicious based on an alert or high suspicion activity that was brought to the attention of an analyst.
While we saw some good examples of correlation in the results (seen in substeps with detection categories tagged with Correlated), the instances in which it was applied only correlated across a single data source. We did not see individual actions correlated across different data sources (i.e., host and network), functional systems (i.e., actions against the safety PLC’s that may impact the control PLC) or observed process values/equipment states. We also did not see any high-level explanations of impacts to the control system. Industry should continue to improve to provide not only visibility, but also greater context to describe the significance of the actions performed against OT networks and control systems. This ability to eloquently convey the story to the end user will drive the industry forward.
Thankfully, we did see elements of the “perfect” solution in some of the participant’s platforms. For instance, we saw timelines of events in certain products that eased the burden of tracking our adversary emulation in a single view. We saw correlation of Windows host-based data sources which tied together related events associated with the execution of malicious programs on a single asset. We also saw events pulled together into a story format, relating network sessions, and providing root cause analysis. Despite these positive efforts to endow detections with context, we think more can be done, especially when dealing with the detection of standard engineering and management actions.
An Adversary Emulation Consisting of Fairly Standard Actions
Our adversary emulation did not consist of zero-day vulnerabilities and obscure actions. Instead, we leveraged standard management and engineering actions that are used to access and manage systems in the ICS technology domain every day. In our case, however, we used these actions with malicious intent, as an adversary would. Other than identifying small nuances in how we executed these actions, they are not necessarily interesting on their own. It is the context that the detection platforms add to them that makes them stand out and worthy of further investigation.
We’ll quickly break down two key elements we focused on in control system protocol communications to provide some background into the standard actions we used for our adversary emulation. We separate control system protocol actions into two types of communication: management and operational. Management communication encompasses actions that change the configuration of a system, such as a Program Download (T0843) that results in the Modification of Programs (T0889) and Controller Tasking (T0821). On the other hand, operational communication includes aspects of control system protocols that allow for the monitoring and control of industrial equipment. This can include techniques such as Unauthorized Command Message (T0855), which describes the usage of protocol communication to actuate equipment. Both management and operational communication can be used to create impacts such as a Manipulation of Control (T0831) or Loss of Safety (T0880), albeit in different ways.
Nothing makes these standard actions bad in and of themselves unless you can show that an action will negatively impact the control system, should not have been performed, or is directly related to some other malicious activity. If you don’t have context into why these actions are bad, then you will just record and bring attention to actions across an environment which may or may not be of use to the end user. In some environments, management communication happens so infrequently that tracking and alerting on these actions makes sense, whereas in other environments it happens so much that it loses its meaning from a threat perspective. This is why the aforementioned contextual information surrounding these actions is essential.
While telemetry is great for ensuring that information is not lost and higher-level detections are generated based on a solid foundation, without proper context, these granular events are hard for the average user to process. This is especially true when the telemetry is voluminous, and you don’t know exactly what you’re looking for. Even singular analytic detections lose their meaning when presented in isolation. Valuable context can be built by tying detections together in a coherent fashion to tell a bigger story about how behaviors not only relate to each other in time but cause other events to happen. The evaluation captured the platforms’ ability to detect individual actions, add some level of context to these actions and, in some cases, correlate across singular data sources. We believe industry still needs to improve upon its capability to correlate across different data sources (i.e., host and network), functional systems (i.e., actions against the safety PLC’s that may impact the control PLC) and observed process values/equipment states. Similarly, more focus needs to be placed on explaining the impacts the actions have on control systems.
This all points to the lack of a demonstrated capability to truly provide context to the entirety of actions that were performed against the evaluation environment. The closer we got to the adversary’s end-goal of physical destruction of the plant, the less contextual information we were provided as analysts and the detections of the platforms’ moved into the telemetry and none detection category region. Now this may be ok if we’re prioritizing early detection, response, and remediation. Nobody wants an adversary to get that far into their system anyway. But if the vision is that OT analyst will be looking at these platforms to identify and respond to threats, these standard actions are going to end up being misunderstood and ignored without proper context.
A Key Step to Providing More Contextual Awareness to Analyst
There are opportunities for industry to provide additional valuable context around detections to improve their usefulness. One of these opportunities involves the correlation of malicious activity across different types of data sources. Throughout the adversary emulation, we created substeps that related to the same action but required different data sources to detect.
For instance, substeps 4.B.1, 4.B.2 and 4.C.1 are all related to plink.exe being used to create an outbound tunnel for command and control. Detection of substep 4.B.1 and 4.C.1 rely on Windows host based data and the ability to detect that a file called SMBClient.exe is Masquerading (T0849) as plink.exe and that there is a network connection established on port 3389. Detection of substep 4.B.2, on the other hand, relies on network data and the ability of the detection platform to not only detect a network session over 445 but to also identify a protocol mismatch indicating SSH tunneling over a port historically used in the evaluation environment for SMB communication. Although the pieces were there, we did not see any detections that stated that behavior was observed indicating network restriction bypass through RDP tunneling based on the executable, SMBclient.exe(masquerading as plink.exe), being used to create an SSH tunnel over port 445. We think that this is valuable analytic work that platforms like the ones evaluated should be able to accomplish and convey to an analyst.
Another example where it is possible to provide correlation across host and network-based data sources can be seen across substeps 19.B.1, 20.A.1 and 20.A.2. In these substeps we see file creation of executables masquerading as Allen-Bradley applications, the execution of one of these files on the safety engineering workstation and a Get Attribute Single CIP request going across the wire to the safety PLC. We didn’t see any demonstration of the platforms’ ability to correlate these detections across different data sources and present the information in the UI. Again, the pieces were there in most cases but we did not see any detections that stated that behavior was observed indicating that a newly created file, RSLogix5000, is masquerading as an Allen Bradley executable and issued a CIP request for a “Status” attribute over the network.
Both of these examples show that while detections on atomic behaviors may have some value, bringing together the multiple data source perspective provides analyst a great deal more context. It is our hope that industry continues to find ways to fuse this type of information together.
Conclusion
Asset owners face a key challenge in hiring and retaining staff with the necessary skills to monitor OT networks. Detection platform vendors can drastically aid in alleviating this skills gap by adding more context to the voluminous telemetry data generated by their detection platforms. Context can provide greater awareness as to why a detection is relevant and how it relates to other events captured in the platform. This could be a force multiplier in the industry’s quest to effectively monitor Operational Technology (OT) networks.
While detection categories, and most notably the Correlated modifier, give insight into how vendors provide context today, they also highlight room for improvement. Most notably, we hope to see solutions continue to pull together data from multiple sources and provide greater context to describe the significance of discrete detections and how they relate to an OT network, control system and industrial process. The ability of these solutions to address the need for added context will determine the overarching success or failure for the solutions to defend these environments.
If you have any feedback or are interested in participating in a future ATT&CK Evaluation, please reach out to evals@mitre-engenuity.org.
© 2021 MITRE Engenuity. Approved for Public Release. Document number AT0021.
