Understanding the MITRE ATT&CK APT29 Evaluation Results:
Cutting Through the Marketing Buzzwords
NOTE: This article has been translated into English from the original Medium article in Chinese.
Foreword
Birdman Chiu, CyCraft Founder & CTO
MITRE ATT&CK® Evaluations use known attack methods of infamous APT hacker groups to test security products. All evaluation results are made public, providing complete transparent evaluation data of the 21 participating vendors. While vendors can interpret and market the data on their own, MITRE provides no rankings or scores. The transparent results include screenshots of all 21 vendors’ solutions, providing users with a valuable under-the-hood view of these security products.
From the beginning, I felt that participating in the MITRE evaluation would be an important opportunity for CyCraft. In Taiwan, too many vendors are presenting POCs for their solutions to users who don’t fully understand cybersecurity products. Users only rely on visually alluring brochures, outdated unsystematic evaluations, and old-fashioned snake-oil huckstering. The most common of which comes across like a modernization of cricket fighting, which was once popular in Southern Taiwan decades ago.
In a cyber cricket fight, a potential buyer puts 10 malware onto a machine, has each vendor individually scan the machine, and goes with the EDR vendor with the highest detection rate. These cyber cricket fights illustrate how Taiwan users have a long way to go in modernizing, not just solutions but evaluations as well.
It doesn’t help matters that some could argue cybersecurity products are difficult for users to understand and evaluate sufficiently, and they wouldn’t be completely wrong. In order to help Taiwan rid the stigma of these outdated cyber cricket fights, we resolved to participate in the MITRE ATT&CK Evaluations despite being the youngest and the second smallest of all 22 vendors to have ever joined a MITRE ATT&CK Evaluation.
I wanted to bring awareness of MITRE ATT&CK to Taiwan and bring awareness of Taiwan to MITRE ATT&CK. Fortunately, the Taiwan and international cybersecurity communities were both pleasantly surprised by our results.
Before starting, I strongly recommended you read the ATT&CK® Evaluation results, overview, and detection categories first. Due to the quarantine and social distancing restrictions, we know your motivation might be lacking, or you just might not completely understand a few things. We wrote this brief introductory guide to shed light on what we can and help clear up any misunderstandings you might have.
In recent years, the international cybersecurity community has rapidly adopted MITRE ATT&CK’s framework, terminology, and evaluations. At the latest RSA Conference, there were countless talks on ATT&CK and how various security vendors respected the MITRE ATT&CK framework and evaluations, how the results affected their internal policies, and how they have implemented the framework into their solutions. However, in Taiwan, there are few resources in Chinese on ATT&CK. There were only a small series of reports from iThome (here and here) and a few Henry Hu articles.
CyCraft is the first Taiwan cybersecurity vendor to participate in the MITRE ATT&CK Evaluations, so it’s our responsibility to take what we have learned from both our experiences with MITRE and our real cyber combat experience, to promote MITRE ATT&CK and educate local users and vendors. Due to the NDA we signed with MITRE, we cannot discuss certain aspects of the MITRE ATT&CK Evaluation methodology and process. Therefore, the material in this article is primarily based on the information published on the official MITRE ATT&CK APT29 Evaluation results page.
Now, John Jiang, CyCraft Senior Endpoint Security Researcher, will guide you through the MITRE ATT&CK APT29 Evaluation Results :)
Understanding the Pyramid of Pain
The Pyramid of Pain Model describes both the pain points of the attacker and what level of defense the defender can adopt to inflict more pain upon the attacker. The higher a defender goes on the pyramid, the harder it is for an attacker to breach their system. At the very top of the pyramid is TTP.
The MITRE ATT&CK Framework mainly describes the attacker’s tools and TTP (tactics, techniques, and procedures). The more a defender makes good use of the ATT&CK framework as a detection benchmark, the more pain will be inflicted upon the attacker.
This is why the ATT&CK Framework has been highly valued by the global information security community in recent years. Although some security products, such as detection and blocking, still rely on IoCs (Indicators of Compromise, e.g., Domain, IP, Hash), in the long run, the real ability to capture hackers still resides in understanding their TTP.
For example, a lot of anti-virus software can detect Mimikatz and its variants, but when a fileless version of Mimikatz or one of its variants appears, some solutions have difficulty detecting it because they’re scanning for the IoCs and not the TTP of this particular attack method.
Not All Techniques Have to Be a One-Hit Kill
Before officially looking at the Evaluation results, you must understand the following:
ATT&CK classifies many techniques used by attackers; however, ordinary people may also legitimately use these techniques in daily operating system maintenance (e.g., T1083 File and Directory Discovery). For example, people often use commands like dir or ls.
Even though a vendor may have integrated the ATT&CK Framework terminology into their solution’s platform or alerting system, an adversarial technique detection may not necessarily be malicious in nature.
Sometimes, it could be a clear indicator of an attack method (e.g., T1003 Credential Dumping), and sometimes it should be regarded as supplementary information (e.g., T1083 File and Directory Discovery). In either case, it should be the goal of every solution to provide analysts with enough contextual information so that each alert is clear and actionable.
Some Techniques Are Difficult to Detect by Rule-Based Detection
Some simple attack methods (“adversarial techniques” to use ATT&CK terminology) can easily be detected through a log or an IoC without additional information or context. Therefore, adversarial techniques could be divided into two types. The first type could be identified by a simple IoC, and the second type would require contextual information of attack behavior.
Attackers have many ways of implementing adversarial techniques, so some techniques are challenging to detect with 100 percent confidence. For example, the following two techniques are not simple attack methods that can be identified by an IoC.
These techniques provide end users with an excellent opportunity to observe the efficacy of security products. Solutions with better efficacy should be able to detect these techniques with 100 percent certainty.
T1122 Component Object Model (COM) Hijacking
There are too many COM components that may be hijacked, and new software installations may show similar signs of behavior. Typically, the detection of whether malicious hijacking is encountered or not requires contextual information and logical judgment. To use a laymen example, a person with a sharp object on a plane does not necessarily lead to a hijacking, but if they used a sharp object in close proximity to or to threaten the crew, this would most likely lead to a hijacking. This technique requires context and understanding of the situation to clearly determine whether it is a hijacking or not. According to the APT29 Evaluation results, none of the vendors’ detections could provide enough contextual information to receive the Technique classification for their detection, if they detected it at all.
Read the lsass.exe memory. Dump the NTDS database of the domain controller. Read the Credential in the SAM. These are all the same adversarial technique. When multiple attack methods can achieve the same technique, nobody can guarantee this detection with 100 percent accuracy and confidence, such as in substep 6.A.2 where APT29 executed the CryptUnprotectedData API call to decrypt Chrome passwords.
Evaluation
The MITRE ATT&CK evaluation team is responsible for the red team attack, verifying the blue team (the vendor participants) detections, and assigning detection category labels and modifiers. The MITRE ATT&CK Evaluations are most admired for their full disclosure of test results and product images of participating vendors, bringing the benefit of full transparency to the public and end users of cybersecurity products.
APT29 Evaluation Detection Categories
During the red team-blue team test, the ATT&CK evaluation team will assign each test item one of the 5+1 detection categories and 0 to 6 modifiers.
The “+1” detection category is MSSP (Managed Security Service Provider). MSSP is done remotely and is later delivered to MITRE (note that MSSP was optional, so not every vendor participated in the MSSP testing), hence MSSP being referred to as “+1”.
There are 5 categories that occur on-site (Technique, Tactic, General, Telemetry, None). The detection result of the vendor must echo the content of the attacker’s test item. The evaluation team will then assign the appropriate detection category label. If the vendor cannot detect the test item, or the detection result content is not related to the attacker’s test item, the evaluation team will assign the None (undetectable, or not recognized) detection category.
Therefore, every detection result is verified by the ATT&CK evaluation team, so false positives do not exist within the MITRE ATT&CK Evaluation results; however, the solutions from the vendors could still be capable of producing false positives. It just wouldn’t be represented within the Evaluation data.
Thus, each effective detection is assigned 1 of 4 effective detection categories in the evaluation: Technique, Tactic, General, or Telemetry. For the three higher-level detection categories of Technique, Tactic, and General, ATT&CK may assign the “alert” modifier label.
Therefore, “detection” and “alert” are different concepts in terms of the MITRE ATT&CK APT29 Evaluation. “Alert” is a modifier (sub-category) of a “detection”.
The “alert” modifier does not apply to Telemetry detections, which is why General, Tactic, and Technique detections have a red background in the following infographic from MITRE.
In order for everyone to have a clearer understanding of how the detection categories are assigned, let’s look at APT29 SubStep 3.B.2 as an example. We’ll view the same SubStep of the attack through the context of each of the 5+1 detection categories. Please remember that the same attack methods were used to evaluate all 21 participating vendors.
Attack Method: Privilege Escalation executed via PowerShell payload
Evaluation Standard: Vendor can see with a high integrity level that PowerShell.exe was executed from control.exe, which was executed from sdclt.exe.
Detection Classified as Technique
Attack Detected
The vendor automatically provides a description of the attack substep after processing and analyzing the raw attack data. In addition, the vendor also corresponds the attack data to one of the defined ATT&CK Adversarial Techniques from the MITRE ATT&CK Framework or describes the attack step with an equivalent description of the Adversarial Technique. In addition to detecting the ATT&CK Technique Privilege Escalation, the vendor also provides enough detail about the attack method, in this case, Bypass UAC. “Technique” is the detection category with the most contextual and accurate information.
Example:
A Technique Detection marked ATT&CK Technique “Bypass UAC”. This detection was also correlated to the source user and executed rcs.3aka3.doc.
Detection Classified as Tactic
The vendor detected the attack but failed to show knowledge of the exact attack method.
After processing and analyzing the raw attack data, the vendor automatically provides a description of the attack substep that corresponds the attack data to one of the defined ATT&CK Adversarial Tactics from the MITRE ATT&CK Framework or describes the attack step with an equivalent description of the Adversarial Tactic. As long as the ATT&CK evaluation team could see the vendor accurately described TA0004 Privilege Escalation, the detection would be classified as a Tactic Detection.
Example:
A Tactic Detection detected Privilege Escalation via PowerShell but did not indicate the specific method, Bypass UAC.
Detection Classified as General
Malicious behavior detected.
After processing and analyzing the raw attack data, the vendor automatically provides a description of the attack substep. While no ATT&CK Technique or Tactic is explicitly indicated, malicious behavior is automatically detected and marked.
Example:
A General Detection discovered malicious PowerShell execution and marked malicious PowerShell with different colored windows.
Detection Classified as Telemetry
Telemetry is low-level data — recorded raw data.
The original raw attack data has not been processed or analyzed; there is no alert. The vendor provides related data regarding the attack substep currently being tested. The vendor only provides simple labeling without further logic or processing. Compared to Technique Detections, Telemetry is the detection category with the least accurate and contextual information. Telemetry Detections are typically part of the data presented on the UI or even the original log inside the manufacturer’s product, such as a DB field, XML, or JSON.
Example:
A screenshot of the vendor’s product shows that powershell.exe was launched control.exe.
MSSP Detections (Optional)
MSSP Detections are presented from an MSSP (Managed Security Service Provider) perspective. Security analysts, solely through human analysis, may participate remotely (after the actual evaluation). These MSSP reports manually interpret the tested emulated attack from human memory — no raw attack data is given to the vendors. Writing comprehensive MSSP reports requires not only technical knowledge but also technical writing skills.
The MSSP detection category is more of an evaluation of the technical knowledge and comprehensive service capability of a vendor’s security analyst team as opposed to the vendor’s actual cybersecurity products or solutions.
Update:
Frank Duff announced in his Feb 21 blog that the MSSP category will no longer be included in MITRE ATT&CK Evaluations, starting with Round 3 against Carbanak + FIN7.
Detection Modifiers
MITRE ATT&CK used a total of 7 modifiers (later reduced to 6) to bring context to each of the detection categories. These 7 modifiers were Alert, Correlated, Delayed, Host Integration, Residual Artifact, Configuration Change, and Innovative.
The “Alert” modifier is only assigned to General, Tactic, or Technique Detections. “Alert” is not a modifier of Telemetry Detections.
The modifiers “Host Integration” and “Residual Artifact” are only assigned to None Detections.
The modifier “Configuration Change” was further subdivided into two categories describing the nature of the configuration change: UX Config Change or Detection Config Change.
The modifier “Innovative” was not used at all.
The Alert Modifier (To Immediately Notify Security Analysts Upon Detection, or Not)
In the APT29 Evaluation, MITRE ATT&CK defined the detection modifier “Alert” as data “presented as priority notification to the analysts as an indication of a suspicious or malicious event occurring for further investigation” via an icon, queue, highlight, popup, etc.
In the APT3 Evaluations, the evaluation term “Tainted Detection” caused some confusion. This time, I think “Alert” can be listed as the most misunderstood modifier in the APT29 Evaluations.
“Technique, Tactic, General Detections may not have Alerts, but if there is an Alert, then it must be a Technique, Tactic, or General Detection.” That’s not an easy sentence to wrap your head around and has undoubtedly been the cause of some confusion.
The word “Alert” carries innate connotations that don’t fully correlate to how the APT29 Evaluations defines and uses the word “Alert”. You might hear this, “Too many alerts could be a result of too many false positives,” but remember, there are no false positives in the ATT&CK Evaluations data.
In regards to Alerts, vendors participating in the APT29 Evaluations can be divided into two groups:
Vendors who had an “Alert” modifier for every Technique Detection:
CyCraft, Palo Alto Networks, Microsoft, FireEye, Cylance, SentinelOne, and others.
Their design concepts are more similar. Regardless of the Technology or Tactic detected, these labels are given to the analysts for reference, so in this APT29 Evaluation, these detections will all have the “Alert” modifier.
After my analysis, it was interesting to discover that 12 vendors followed this model for the APT29 Evaluations — just slightly more than half. Exactly how each of these vendors actually alerts their users when they are in serious threat will vary. Some vendors send a message to users when the threat level of the Technique Detection is above a certain threshold or degree, while alerts with a low threat level are recorded and regarded as supplemental data for further future analysis.
Vendors who did not have an Alert Modifier for every Technique or Tactic Detection:
CrowdStrike, TrendMicro, Symantec.
Their design concepts deal more with grading alarms — triage. If there is sufficient data, the solution will notify the end-user security personnel. If there isn’t sufficient data (even if the telemetry data contains supplemental information of the Adversarial Technique or Tactic), the end-user security personnel will not be notified.
MITRE separates the concepts of “detection” and “alert” by using the “Alert” modifier to confirm that one particular detection triggered the solution to notify the security personnel and made them aware of the detected step of the current cyberattack.
12 vendor solutions notified their end users for each Technique Detection; 9 vendors didn’t. More than 50 percent of the time, Symantec and TrendMicro did not notify their end users of the Technique Detections made by their solution.
However, this data, as well as the MITRE Evaluation, does not go into further detail or evaluation whether or not these vendors’ solutions have features that may correlate these Technique Detections at a later time other than when these Technique Detections were immediately detected.
Being aware of these two different design strategies is important and will hopefully bring clarity to understanding the usefulness of the “alert” modifier when analyzing a vendor’s results.
There is no certain standard for which solution is better. In the later “How to Use ATT&CK Evaluations” section of this article, I will discuss how to select a vendor with this information.
The Correlated Modifier (Immediately Relating Detections to Each Other)
This modifier is relatively straightforward. As long as the Detection has sufficient data to relate the previous attack step and immediately notifies the end-user security analyst, the Detection will be given the Correlated modifier.
Example:
In our cyber situation graph, you can easily see that sdclt.exe was launched by cmd.exe, and cmd.exe was launched by rcs.3aka3.doc. As this can be related to the attacker’s previous attack step, this detection was given the “Correlated” modifier.
The Configuration Change Modifier (On-Site Configuration Required to Validate Detection)
This modifier is divided into two types: UX Change and Detection Change.
UX Change: At the time of testing this particular attack step, the vendor determined that the product had recorded the attack-related data, raised these concerns to the evaluation team, and adjusted the product interface-related settings to display relevant data to the user on the interface. The attack step was detected, but for whatever reason, the detection did not appear on the UI.
Detection Change: At the time of testing this particular attack step, the vendor’s solution could not detect the attack step. After learning what attack method the attacker (the MITRE ATT&CK Evaluation team) used, the vendor deployed new processing logic or adjusted their sensor to collect different types of data, and then requested for the attack step to be re-evaluated. After the re-evaluation, the vendor’s solution was capable of providing sufficient data to detect the re-tested attack step and classify its detection as Telemetry, General, Tactic, or Technique.
It needs to be noted that although the evaluation results record the number of changes, the results do not specifically record the settings modified or added by the manufacturer. Analysts, vendors, and end users who look at the APT29 Evaluation results need to keep this in mind when examining detection results.
The Delayed Modifier
Delayed detections delay the presentation of relevant data to the end user. These factors could include detections produced by subsequent or additional processing. Delayed modifiers do not apply to automated data ingestion, routine processing taking a predetermined interval of time, nor connectivity issues unrelated to the vendor’s solution.
The vast majority of both manual and processing delayed modifiers were given to MSSP detections, which are inherently delayed due to the necessary manual analysis.
Processing: The solution analyzes attack data using sophisticated machine learning algorithms that trigger a detection from minutes to hours after the red team performed that particular attack step. The detection could still be labeled as a Technique Detection but would have the delayed modifier.
Manual: The wisdom of the MSSP team. These detections could have been delayed due to manual analysis of the attack data by the analyst team. Like the processing modifier, these detections could still be labeled as a Technique Detection but would have the delayed modifier. (This did not happen during the evaluation for any Technique or Tactic Detections, but did happen for Telemetry and General Detections.) These manual delayed detections also include content generated after the vendor’s MSSP / MDR / EDR team analyzed the attack data and handed in their report.
As MSSP detections will no longer be a part of the MITRE ATT&CK Evaluations, the guidelines and definitions of the delayed modifier may be subject to change.
How to use ATT&CK Evaluations
The MITRE ATT&CK APT29 Evaluations do not provide rankings or scores of vendors or their solutions. While all vendors are subject to the same evaluation restraints and guidelines, some solutions had features not included within the evaluation, such as NGAV, NGFW, or Cyber Situation Reports.
The MITRE ATT&CK APT29 Evaluations do provide a screenshot of the performance of vendors for one particular threat and do provide full transparency of each vendor’s performance.
While the evaluation does give everyone access to the results, it is important that end users fully understand the details behind the detection categories and modifiers to accurately choose the most suitable product for their unique cyber situation and gives them useful background data prior to researching cybersecurity POCs.
Picking the Solution and Vender that Best Suits You
I could write novels about how to choose the most suitable vendor for your unique cyber situation. However, if we reduce our focus to the scope of the APT29 Evaluations, we should first look at the key indicators observed from the evaluation.
Security Defense Strategy:
This concept covers a wide range of strategies, but you should first know where your defenses are currently lacking. What are your primary concerns? What Techniques and Tactics are most harmful to your current defense plan? (You could even create a heat map with ATT&CK Navigator.) Overlay your defensive concerns on top of vendor results. Do these vendors meet your needs?
IoC Analysis Capability:
If you have limited SOC personnel or your SOC capabilities have not fully matured, I recommend you begin researching solutions with API/UI your security personnel are capable of understanding. The MITRE ATT&CK APT29 Evaluation results include screenshots of the 21 vendors’ solutions. The benefit of this evaluation is that your team can filter out solutions they find completely incomprehensible, unintuitive, or not within their defensive needs.
The reason for this assessment is that while many vendors do collect telemetry attack data, the API/UI used to read and analyze the data may be lacking. Reading attack data is quite difficult without annotations. Some data representations are not even designed for human analysis.
Information Security Maturity:
If you are considering MSSP/MDR Services, it is recommended you research vendors who provided MSSP detections during the evaluation. Automated, real-time detections (zero human analysis) are incredibly useful for reducing mean time to detect (MTTD) and mean time to respond (MTTR); however, APT-level cyberattacks often contain zero-day exploits or unseen tools or tactics. Human analysis is needed in these situations to accurately correlate IoCs and bring context to detections. The MSSP detections provide you a metric into the efficacy of a vendor’s human analysis methodology.
Your Cyber Environment Maturity Determines Your Priorities
In order to simplify the complexity of analyzing the evaluation results, we will look at how all 21 vendors performed on one of the more complicated attack substeps — APT29 substep 20.B.1.
20.B.1 Pass the Ticket
Procedure: Created Kerberos Golden Ticket using Invoke-Mimikatz
Criteria: powershell.exe executing Invoke-Mimikatz with command-line arguments to create a golden ticket
“Pass the Ticket” is a classic method of authenticating to a system without having access to an account’s password. It can be used as the first step to laterally move to a remote system, can be tricky to detect, and is one of many ways the Windows AD can be compromised.
Please don’t forget to pay attention to the Configuration Change modifier (the gear icon) during your analysis; it is necessary to consider whether the out-of-box settings are capable of detecting this particular technique. When analyzing individual attack substeps, I recommend using the Technology Comparison tool from the official website.
Example 1: Your Security Maturity is Not High
I am familiar with cybersecurity industry terms. I rely on MSSP and/or MDR. Our team of analysts is limited.
First, pick out vendors with MSSP detections or that provide MDR services.
*Note that six of the 21 vendors did not participate in the MSSP evaluation: Bitdefender, Endgame, Hansight, Malwarebytes, McAfee, and Reaqta.
After glancing at the data provided by the Technology Comparison tool on 20.B.1, try following this line of thought to help narrow down the vendor solutions that best suit your cyber environment.
- Vendors with Technique or Tactic Detections
- General Detections indicate the detection of suspicious behavior
- Vendors who had MSSP Detections or alert modifiers on any GTT Detections
- If a vendor does not have any of the above, strongly consider excluding them from your list of potential candidates.
Why is the MSSP ranked third?
Remember that in this evaluation, each vendor is given information about each attack step as it is occurring, which in some ways is equivalent to knowing the exam answers; you just have to show your work to gain credit. As long as the correct answer is given within a few days after the announcement, it is counted as a detection. A screenshot could be classified as an MSSP. Technique and Tactic detections are ideal. General detections are also acceptable.
Example 2: Your Security Maturity is Above Average
I am a full-time analyst. The more detailed and contextual information you give me, the better! Our SOC has sufficient technical knowledge. Some SOC analysts may want super-detailed information; some consider that installing free tools like Microsoft’s Sysmon can save money.
If you are such a person, try following this line of thought to help narrow down the vendor solutions that best suit your cyber environment.
- Decide which substeps are a priority for you.
- Overlay vendor detection coverage over your prioritized substeps. What vendors provide coverage over your prioritized substeps?
- Use the Technology Comparison tool to focus on substeps your team has prioritized. What vendors provide useful detection data?
- Look into each vendor’s individual results and find a product interface your team likes and can understand.
For cybersecurity enthusiasts, the third point is particularly important. If you only see the commands (process cmd-line, powershell cmd-line, etc.) but do not see the system-level behavior (API Call, file event, etc.), it is relatively meaningless for data collection.
Example 3: SecOps, High Maturity
I am an information security operator, and I will be notified if there is a clear warning. My time is precious!
This type of person cannot analyze raw data in front of the computer for 24 hours. They need to triage their time on other security operations.
The difference between this type and the first example is that the SecOps is specialized security personnel and directly deals with security incidents. In reality, their time is limited. They must focus on validated alarms. There is no time to analyze the collected raw telemetry data.
In this kind of situation, try following this line of thought to help narrow down the vendor solutions that best suit your cyber environment.
- Identify vendors that can provide MSSP / MDR as their solutions may be better at triage.
- Identify vendors that can generate more Technique and Tactic detections as these detections can greatly reduce MTTD and MTTR.
- Use the Technology Comparison tool to focus on substeps your team has prioritized. What vendors provide automated General, Technical, and Tactic detections?
- Identify a product interface that your team likes and can understand.
Conclusion
The MITRE ATT&CK Evaluations don’t just lift the curtain behind vendors’ solutions but also lets us know which solutions are best suitable for our unique cyber environments. At the same time, by giving various vendors a safe environment to compete, they can identify weaknesses and better tune their solutions. Users are able to see which vendors continue to make progress and which vendors have gained confidence through their experiences, both in evaluations and in the wild.
However, due to MITRE ATT&CK’s unique evaluation method, vendors are able to represent the result data from a variety of viewpoints — typically the viewpoint that puts them in the best light. Although analysts have had issues with the detection categories and modifiers of the second round of evaluations, most do agree that the labeling method from the first round of evaluations has been greatly improved upon.
To be honest, the entry points of the 21 cybersecurity vendors are quite different, so the design focus of each solution has its own strengths; one vendor, for example, specializes in preventative solutions (e.g., ACL, FW, AV) — not really fair to compare it with an EDR or MDR solution. The red team-blue team step-by-step evaluation method should be recognized by the industry as the most fair among the unfair, not to mention that MITRE ATT&CK is your opposing red team. I wrote this article in hopes that local vendors gain insight into how MITRE ATT&CK runs their evaluations, local users gain insight on how to be more discerning when selecting security solutions, and that local users and vendors can both realize the detrimental nature of “cyber cricket fights” where the best POC wins and modernize their evaluations.
Who’s the real winner of the MITRE ATT&CK APT29 Evaluations?
The users.
Your MITRE ATT&CK Reading List
1. Introduction | What is MITRE ATT&CK?
2. Behind the Curtain | Who is MITRE?
3. ATT&CK Evaluations Round 2: APT29 | CyCraft Enters Round 2
4. ATT&CK Evaluations Round 2: APT29 | Complete Guide to Understanding Results
5. ATT&CK Evaluations Round 2: APT29 | CyCraft Results
6. ATT&CK Evaluations Round 3: FIN7 and Carbanak | CyCraft Enters Round 3
Postscript:
During the 2020 RSA conference (if you can remember back to those pre-pandemic days), MITRE ATT&CK arranged a special night where they invited all the vendors who participated in the evaluations to gather for celebration and debate. Our entire team at CyCraft wants to thank MITRE ATT&CK for the great night and the opportunity to crush at the evaluations. It’s not often where the youngest pup in the room gets a chance to run with the big dogs.
Follow Us
When you join CyCraft, you will be in good company. CyCraft secures government agencies, Fortune Global 500 firms, top banks and financial institutions, critical infrastructure, airlines, telecommunications, hi-tech firms, and SMEs.
We power SOCs with our proprietary and award-winning AI-driven MDR (managed detection and response), SOC (security operations center) operations software, TI (threat intelligence), Health Check, automated forensics, and IR (incident response), and Secure From Home services.
Additional Related Resources
- Using ATT&CK for CTI Training | MITRE ATT&CK®
- ATT&CK Evaluations: Understanding the Newly Released APT29 Results
- Quantifying the MITRE ATT & CK Round 2 Evaluation
- CyCraft CEO, Benson Wu, and CyCraft Global Project Manager, Chad Duffy, speak on the latest MITRE ATT&CK Evaluations. Read their thoughts on our results and the philosophy powering CyCraft.
- Learn how we detected and defeated a foreign APT targeting Taiwan’s high-tech ecosystem. Read our full analysis and malware reversal.
- Has your organization recently shifted to a Work From Home environment? Learn how to receive 3 FREE months of our Secure From Home Service.
- Our Enterprise Health Check drops your mean dwell time down from 197 days to under 1 day without false positives or false negatives. Know with confidence if hackers have penetrated your enterprise.
