Data that would shed light on claims of Russian meddling in the 2016 U.S. Presidential election remains out of investigators’ hands.
Part One: Cozy Bear and Fancy Bear
To hear Dmitri Alperovitch tell it, the moment had all the tension of a Hollywood blockbuster: a phone call in the early morning hours, a quick exchange of words, and a sudden, dramatic realization — the Democratic National Committee was under attack. “Are we sure it’s Russia?,” Alperovitch asked the security analyst on the other end of the line. The analyst, a former intelligence officer trained in the art of cyber warfare, told Alperovitch that there was no doubt.
Alperovitch is the 37-year-old chief technical officer of the cyber security company CrowdStrike. CrowdStrike’s proprietary cyber security software, named Falcon Host, a “next generation” endpoint security technology, had been installed on the servers of the Democratic National Committee just the night before, in response to a suspected intrusion. Within “ten seconds” the software signaled a positive hit — the malware that had been detected by Falcon had the same traits as those previously used by two “advanced persistent threats,” or APTs, known to CrowdStrike by the code names Cozy Bear and Fancy Bear.
Cozy Bear was assessed by Alperovitch to be run by the Russian Federal Security Service, or FSB; Fancy Bear was attributed to Russian military intelligence, or the GRU. CrowdStrike’s Falcon software showed that Cozy Bear had been active in the DNC server since the summer of 2015, mapping out directories and exfiltrating data. Fancy Bear had penetrated the DNC server in April 2016, and had stolen some files related to opposition research.
Alarmed by what Falcon had uncovered, Alperovitch made a phone call to Shawn Henry, a former senior FBI official who had headed CrowdStrike’s incident response capability since being recruited straight from retirement in 2012. Henry and his team deployed an intelligence tool, Falcon Overwatch, to assist in what CrowdStrike terms “the incident response engagement.” According to CrowdStrike, Falcon Overwatch is a 24/7 global operations center staffed by “an elite group of cyber intrusion detection analysts and investigators” who hunt for adversary activity and malware in a client’s server.
“The OverWatch team saw activity as it occurred with ‘over the shoulder’ observation of ‘hands on keyboard’ as command line activity occurred,” Henry recalled in a company after-action report. “This real-time information provided the CrowdStrike Services team with additional indicators of compromise to examine, which in turn helped reveal what the attackers were trying to accomplish.”
CrowdStrike has a corporate motto: “You don’t have a Malware Problem, You Have an Adversary Problem.” The ethos behind this motto, as explained by Adam Meyers, who runs CrowdStrike’s Global Threat Intelligence Team, is the notion that by focusing on the adversary “you are dealing with the problem, not just a symptom of the problem. Malware deals with the symptom.” Adversaries, Meyers notes, “are the humans behind the attacks. We spent years in security focusing on malware and exploits and techniques, but not on who is perpetrating them. There are humans behind the attacks, so we watch for patterns, use intel to zero in on the human element. We ask who they are, what their motivation is and what types of things they are likely to do in the future.”
On April 29, 2016, when the DNC became aware its servers had been penetrated, an emergency meeting was held between the Chairwoman of the DNC, Debbie Wasserman-Schultz, DNC’s Chief Executive, Amy Dacey, the DNC’s Technology Director, Andrew Brown, and Michael Sussman, a lawyer for Perkins Coie, a Washington, DC law firm that represented the DNC. Sussman took control of the meeting, setting out the DNC’s agenda when it came to dealing with the cyber attack on its server. The three most important questions, Sussman declared, were what data was accessed, how was it done, and how can it be stopped?
The one question Sussman, a former federal prosecutor who focused on computer crimes, did not ask was, who did it?
It took the DNC four days to decide to bring in an outside vendor to investigate the breach of its servers. In the end, it was Sussman who made the call to Shawn Henry at CrowdStrike. The call was made on May 4; by May 5 CrowdStrike had installed its FalconHost software that had triggered the Russian attribution.
This wasn’t the first time CrowdStrike had been called in by the DNC. In December 2015 it tapped the company to conduct an audit of the circumstances surrounding a breach of security involving the DNC’s party-administered voter file system — specialized software developed by the company NGP VAN known as VoteBuilder. Over the course of five weeks, CrowdStrike examined administrative logs from the DNC to assess user activity within the VoteBuilder system, and conducted a forensic examination of two other systems belonging to the campaign of Vermont Senator Bernie Sanders. The results of the CrowdStrike investigation were released on April 29, 2016 — the same day the breach of DNC servers was detected.
Acting on FalconHost’s May 5 alert, CrowdStrike poured over the data. FalconHost had found indicators — malware, techniques, and patterns of behavior — that suggested two APT’s, Cozy Bear and Fancy Bear, were behind the cyber attack on the DNC. Shawn Henry now deployed CrowdStrike’s Overwatch capabilities to answer the questions Sussman had asked: What data had been compromised, how did this compromise occur, and how could the DNC prevent future compromise?
But CrowdStrike had to proceed carefully. If the humans behind either Cozy Bear or Fancy Bear detected that their respective hacks had been discovered, they would be able to cover their tracks and, worse, burrow so deep into the DNC server system it could never be deemed secure.
CrowdStrike, however, had built its corporate reputation on being more proactive than reactive when it came to cyber security. Using what it called Enterprise Adversary Assessment, CrowdStrike sought not only to locate the perpetrator of the attack, but also track them back into their systems and prevent them from ever returning. One of the tools CrowdStrike advertised in this regard involves so-called “attractive data” that had been tagged with tracking malware and placed in the compromised server. Once the hacker fell for the trap, and exfiltrated the file in question, CrowdStrike would be able to follow it back through the adversary’s network, helping build a picture of the adversary’s infrastructure, and hopefully identify what data had been previously stolen and where it resided. Crowdstrike also advertised offensive strategies designed to limit the number and severity of future attacks by disrupting the attackers’ infrastructure using undisclosed techniques.
The services described would seem to run afoul of the Computer Fraud and Abuse Act, or CFAA (under the CFAA, private entities are restricted from pursuing offensive cyber security measures designed to infiltrate an attacker’s computer infrastructure for the purpose of learning how a cyber attack occurred.) CrowdStrike sought to mitigate any legal exposure presented by its new, innovative approach to cyber security by bringing Steven Chabinsky, a former FBI legal specialist with considerable experience in cyber intelligence operations, onboard as its general counsel.
Chabinsky didn’t believe that the CFAA represented an insurmountable obstacle to CrowdStrike’s aggressive approach; if a business was hacked, he held, it had every right to either delete or encrypt its property if it was discovered on the attacker’s computers. Chabinsky’s approach echoed the mentality of Alperovitch, who likened CrowdStrike’s offensive posture to defending ones property rights. “If I tackle you on the street, that’s assault and battery,” Alperovitch noted. “But if a few minutes prior you had taken my wallet, it’s completely legal.”
Shawn Henry and his team used CrowdStrike’s Falcon Overwatch capability to monitor the DNC’s compromised servers for more than 30 days, mapping out the scope of the intrusion and tracking the actions of the attackers. The scope of the Cozy Bear intrusion was potentially devastating. According to CrowdStrike, Cozy Bear had roamed uncontested throughout the totality of the DNC server, collecting and transmitting email and Voice over Internet Protocol (VoIP) communications. Significant amounts of data had been exfiltrated during this time, CrowdStrike assessed, and the DNC had to assume that anything stored in the server had been compromised.
Fancy Bear appeared to have more limited objectives. Henry’s team detected evidence of a few select files having already been exfiltrated, while others were staged for future exfiltration. An analysis of these files showed that Fancy Bear was focused on opposition research being done by the DNC on the erstwhile Republican nominee, Donald J. Trump.
While the CrowdStrike analysts believed they were able to isolate the malware, tools and techniques used by both Cozy Bear and Fancy Bear to facilitate the theft of DNC data, they were not able to determine the source of the initial intrusion for either threat actor. Threat intelligence from previous cyber attacks on other targets (including the German Parliament, a French television channel, TVMonde5, the US State Department and the White House) attributed to both Cozy Bear and Fancy Bear, suggested that the vector used to facilitate initial penetration of a targeted server was through a technique known as a “phishing” attack, where the attacker used fake documents and communications to trick the target into clicking on a field infected with malware. There was, however, no evidence on the DNC server that showed it had been subjected to a “phishing” attack. How the Cozy Bear and Fancy Bear malware came to infect the DNC server remained a mystery to CrowdStrike.
At some point, the decision was made by the DNC and CrowdStrike to go ahead and regain control of the DNC servers. But to CrowdStrike, this wasn’t enough. Sifting through the data collected by Shawn Henry and his Falcon Overwatch team, Dmitri Alperovitch was taken aback by the sheer audacity of what had transpired. Michael Sussman, the DNC legal counsel, agreed. “You have a presidential election underway here and you know that the Russians have hacked into the DNC,” Mr. Sussman told the New York Times. “We need to tell the American public that. And soon.”
At first the DNC tried to get the FBI to make the attribution call, figuring that it would garner more attention coming from the US government. But when the FBI wanted full access to the DNC server so that it could conduct a full forensic investigation, the DNC balked. Instead, after meeting with Alperovitch and Henry, the DNC and CrowdStrike devised a strategy to take the case to the public themselves. Alperovitch prepared a formal technical report that singled out the Russians for attribution. When it was ready, the DNC invited in a reporter from the Washington Post named Ellen Nakashima, who was given exclusive access to senior DNC and CrowdStrike personnel for an above-the-fold, front-page article.
Before the Washington Post could go to print, however, CrowdStrike needed to evict Cozy Bear and Fancy Bear from the DNC server, and deploy security mechanisms designed to keep them out. Over the course of two days, from June 10–12, CrowdStrike stealthily replaced the DNC’s software, moving carefully to avoid detection. With the DNC server clean and secure, the plan to “name and shame” Russia could go forward.
The Post article, published on the morning of June 14, 2016, went viral, with nearly every major media outlet, including the New York Times, citing it in their own subsequent investigations. When CrowdStrike published its technical report 30 minutes later, it was received by a media already driven to a frenzy and starving for information. The report, “Bears in the Midst: Intrusion into the Democratic National Committee,” quickly became headline news, and Dmitri Alperovitch, its author, a household name. The DNC and CrowdStrike, it seemed, had executed the perfect attribution campaign, creating a perfect storm of political intrigue and spy-versus-spy narrative that the media couldn’t ignore.
Part Two: Shady Rat
The public attribution campaign targeting Cozy Bear and Fancy Bear wasn’t the first time Dmitri Alperovitch had engaged in a highly publicized “name and shame” operation. Back in 2011, when Alperovitch worked as the vice president of threat research for the cyber security giant, McAfee, he had published a similarly politically charged report, “Revealed: Operation Shady Rat.” This report was to push Alperovitch’s attribution strategy, and Alperovitch himself, to the forefront of a national dialogue on cyber security.
According to Esquire, it was Alperovitch’s own analysis of administrative logs from a compromised server that led him to conclude that Shady Rat was the culprit behind cyber attacks targeting dozens of companies around the world. Moreover, Alperovitch believed, this evidence pointed to China as the perpetrator of this hacking campaign. McAfee policy prevented this attribution from appearing in a formal company report, but this didn’t stop Alperovitch from naming names; when Vanity Fair specifically asked about the link between China and Shady Rat in mid-July 2011 (prior to the publication of the McAfee report), Alperovitch reiterated McAfee corporate policy about attribution, before observing, “If others want to draw that conclusion, I would not discourage them.”
In the weeks leading up to the public release of the Shady Rat report, Alperovitch privately pushed his view on Chinese attribution in a series of closed-door meetings with the White House, “executive-branch agencies” (i.e., the FBI and intelligence community), and congressional committee staff. These briefings proved to be a media bonanza for Alperovitch — once the Shady Rat report became public, the White House and Department of Homeland Security were compelled to acknowledge their awareness of the report and the issues it raised. The NSA had been tracking the various entities that comprised Shady Rat for years. However, the manner in which Shady Rat presented to the public helped create the impression that it was Alperovitch, and not American law enforcement or intelligence agencies, that uncovered the threat posed by China and Shady Rat.
As a marketing ploy, the Shady Rat report was pure genius, playing on the nexus of public ignorance and political paranoia that existed in the United States. A perfect example of this was a statement made by the Democratic Chair of the Senate Select Intelligence Committee, Dianne Feinstein, after reading Alperovitch’s Shady Rat report. Feinstein emailed Vanity Fair, who quoted her as saying Alperovitch’s Shady Rat report represented “further evidence that…we need to start applying pressure to other countries to make sure they do more to stop cyber hacking emanating from their borders.”
This was music to Alperovitch’s ears, since “naming and shaming” was a core principle behind his approach to cyber security. “We saw that no one’s really focused on the adversary,” Alperovitch later told Esquire. “No one’s focusing exclusively on how can we actually identify them, attribute them, deter them from taking this action again.” From the Shady Rat experience was born the ethos that later morphed into CrowdStrike’s corporate motto: “You don’t have a malware problem, you have an adversary problem.”
Almost overnight, Alperovitch and Shady Rat had become household names. The media (including Washington Post cyber beat reporter Ellen Nakashima) picked up the story and ran with it, making the linkage between Shady Rat and China an incontrovertible fact in the minds of the American public. It also boosted the stock of Aleprovitch and a colleague of his at McAfee named George Kurtz. Kurtz resigned from McAfee in October 2011, and within a month was brought on by Warburg Pincus, a private equity firm, as an Executive in Residence. Kurtz was slated to head up a new cyber security company, CrowdStrike, which Patrick Severson, a Managing Director at Warburg Pincus, was in the process of underwriting to the tune of $26 million in Series A funding.
Kurtz poached a number of senior executives from McAfee to join him at this new startup company, which he named CrowdStrike. But the jewel in CrowdStrike’s crown was the man around whom the company’s operating ethos would be constructed — Dmitri Alperovich. Alperovich left McAfee in mid-September, and by the end of that month was presenting his vision for cyber security at an event sponsored by Brookings. Alperovich helped co-found CrowdStrike with Kurtz and another McAfee alumnus, Gregg Marston. While Kurtz, Marston and Severson’s names populated the SEC filings submitted by CrowdStrike in December 2011 in regard to its funding efforts, it was Alperovitch’s cache that made it all possible.
Alperovitch’s name was well known, thanks in large part to the momentum created by his Shady Rat report and the accompanying Vanity Fair interview, which was published in September 2011. Brought in as the Chief Technology Officer for CrowdStrike, Alperovitch leveraged the reputation he built on the back of Shady Rat to promote one of CrowdStrike’s initial technological initiatives, Crowdsourced Reverse Engineering (CrowdRe), a free collaborative malware assessment tool. Even here, Alperovitch’s history with Shady Rat could be seen: the CrowdRe functional demonstration used as its case study a malware sample CrowdStrike sourced from what it named “Comment Panda” (the name it gave to the cyber adversary Alperovich claimed to have exposed as a result of his Shady Rat investigation).
This collaborative approach to cyber security was part and parcel of CrowdStrike’s operational methodology. “At CrowdStrike,” Alperovitch told one interviewer, “we look for traces of the adversary and try to find out who the adversary is, what they are after, and what their tradecraft is. We also disseminate that information to enable collective action.” In this, CrowdStrike was no different from other cyber security startups. What separated CrowdStrike from the pack were the pro-active measures Alperovitch promoted in defending against an identified threat. “We should enable the private sector to engage in self-defense in the cyber world, like we do in the physical world,” Alperovitch declared. “Licensed cyber security companies” in the mold of CrowdStrike should be allowed “to take certain actions in defense of a network…if you see your data going to some other network, why can’t you go into that network for the purpose of getting your data back, or take data off that machine to mitigate the damage?”
Alperovitch’s aggressive posturing was soon reinforced when, in May 2012, CrowdStrike hired Shawn Henry, who had just retired from 24 years service with the FBI. Henry’s last position with the FBI was as the Executive Assistant Director of the Criminal, Cyber, Response and Services Branch, where he oversaw the entirety of the FBI’s cyber response capability. Like Alperovitch, Henry was frustrated with the approach being taken by the US government and private industry when it came to responding to cyber attacks against American targets. “I don’t see how we ever come out of this without changes in technology or changes in behavior, because with the status quo, it’s an unsustainable model,” Henry told an interviewer from the Wall Street Journal in March 2012, shortly before leaving the FBI.
Alperovitch’s smarts and Henry’s brawn made for a perfect combination of personalities that enabled CrowdStrike to market its new image as a private cyber intelligence agency, one that, according to a Los Angeles Times company profile, “identifies sophisticated foreign attackers trying to steal US intellectual property and uses the attackers’ own techniques and vulnerabilities to thwart them.” This aggressive posturing proved to be highly effective; in 2013 Crowdstrike was able to secure an additional $30 million in Series B funding from Accel Partners and Warbus Pinkus. Alperovitch was showered with praise, selected by MIT Technology Review as a “Young Innovator Under 35,” and named by Foreign Policy as one of the “Top 100 Global Thinkers” for 2013, largely on the reputation he garnered from his 2011 Shady Rat report.
But while Alperovitch may have charmed the billionaires who were underwriting the CrowdStrike enterprise, several of his fellow cyber sleuths smelled a rat. One of them, Eugene Kaspersky, took the time to put his concerns into writing. Kaspersky is the CEO and founder of Kaspersky Lab, a well-regarded Russian-based cyber security company. “We conducted detailed analysis of the Shady Rat botnet and its related malware,” Kaspersky wrote in an August 2011 blog, “and can conclude that the reality of the matter (especially the technical specifics) differs greatly from the conclusions made by Mr. Alperovitch.” Moreover, Kaspersky stated, “We consider those conclusions to be largely unfounded and not a good measure of the real threat level,” adding that “we cannot concede that the McAfee analyst was not aware of the groundlessness of the conclusions, leading us to being able to flag the report as alarmist due to its deliberately spreading misrepresented information.”
Symantec, the maker of the popular Norton Anti-Virus software, shared Kaspersky’s concerns over Alperovitch’s exaggeration and outright misrepresentation of the Shady Rat threat. Writing for the Symantec company blog, Hon Lau, a security analyst, noted that Symantec had “uncovered what appears to be the same information source about the victims of the attacks that was used by McAfee as the basis of their report. This information is freely available on the attackers’ command and control site, which is a strange oversight considering this type of attack is often described as ‘advanced’ or ‘sophisticated.’”
Lau also undercut Alperovitch’s self-made reputation as a super cyber sleuth. “It turned out,” Lau wrote, “that the attackers not only failed to secure their server properly, they had also installed various Web traffic analysis tools on it too, which is of course useful to the attackers to see how they are doing, but makes our lives easier too when investigating such attacks. For example, on one of the sites we were able to see the statistics about computers contacting the command and control server to download command files. Based on this information, we were also able to determine the organizations affected by this threat.”
Both Lau and Kaspersky discounted Alperovitch’s efforts to attribute blame for the Shady Rat cyber attacks on China. “There has been some discussion,” Lau noted, “of this being a government-sponsored attack. However, the finger can’t be pointed at any particular government. Not only are the victims located in various places around the globe, so too are the servers involved in these attacks.”
Kaspersky echoed this point, stating, “It looks overwhelmingly likely that no state is behind the Shady Rat botnet. How the botnet operates and the way the related malware is designed reveals startling fundamental defects hardly indicative of a well-funded cyber-attack backed up by a nation state.”
Alperovitch had described Shady Rat as “an advanced persistent threat,” a “sophisticated penetration” where “the adversary is motivated by a massive hunger for secrets and intellectual property.” Neither Kaspersky nor Symantec shared this conclusion. “When you consider the errors made in configuring the servers and the relatively non-sophisticated malware and techniques used in this case,” Lau wrote, “one could not call Shady Rat an advanced persistent threat.” Moreover, as Kaspersky pointed out, the IT industry was already fully aware of the Shady Rat phenomenon, “but decided not to ring any alarm bells due to its very low proliferation — as confirmed by our cloud-based cyber-threat monitoring system and by other security vendors. It has never been on the list of the most widespread threats.”
Contrary to Alpertovitch’s claims that Shady Rat was responsible for stealing “secrets and intellectual property,” Kaspersky notes that a review of the logs used by Alpertovitch make clear that “there is no evidence showing what sort of data has been acquired from infected computers, or if any data has been acquired at all.” Lau reached the same conclusion, noting, “What‘s still unclear is the type of information the attackers were targeting.”
In retrospect, Shady Rat appears to have been perpetrated for one purpose — to manufacture a narrative that could be exploited for the personal benefit of Dmitri Alperovitch, George Kurtz and Gregg Marston, the three former McAfee executives who founded CrowdStrike. Alperovitch and George Kurtz had been planning to leave McAfee prior to the Shady Rat report being published. Alperovitch told Esquire Magazine that he accelerated his plans to depart McAfee because of his outrage at being “censored” by corporate executives uneasy over his attribution of China in the report. But this is disingenuous; as Alperovitch related to Vanity Fair during his exclusive pre-publication interview, McAfee policy at the time was not to speculate on what country was behind Shady Rat. It was this long-standing policy, and not any knee-jerk corporate reaction to the Shady Rat report, that drove the top-down request to remove specific attribution from the report.
It appears that Alperovitch concocted the Shady Rat threat from thin air, and then promulgated its existence through private meetings with government officials predisposed to accept any public reporting that sustained the notion of a Chinese cyber threat to the United States. Alperovitch, Kurtz and Marston were more than likely planning what would eventually become CrowdStrike well prior to the Shady Rat report being published — one does not simply attract tens of millions of dollars in investment funding on the fly. The entire Shady Rat enterprise — the report, the secret government briefings, the exclusive high profile article — appeared to be designed to elevate Alperovitch’s public profile on the eve of his resignation from McAfee and the creation of CrowdStrike, a profile Kurtz and Marston were only to willing to exploit. If this was indeed the case, it was at a minimum deceptive marketing, and America fell for it.
There was a fourth McAfee executive who claims he was supposed to be a part of the CrowdStrike venture. Stuart McClure, the current CEO of Cylance, a California-based cyber security company (and competitor of CrowdStrike), claims that he was invited by George Kurtz to join CrowdStrike in early 2012, an offer McClure says he turned down. “I decided I needed to live my life with high integrity and with high-integrity people, so I decided to do this gig (Cylance) on my own.” (Kurtz denies that he offered McClure a position in CrowdStrike.) Normally such claims would be downplayed, especially given the contentious history between the two men, who were colleagues and business partners for 14 years before their falling out. But McClure’s reference to “high integrity,” made in a manner suggesting both Kurtz and the CrowdStrike business venture were found lacking in such by McClure, cannot simply be dismissed in light of the Shady Rat fraud perpetrated by Alperovitch and, by extension, Kurtz.
The observations of Jeffrey Carr, a well-regarded cyber security author, are relevant in this regard. In a blog posting titled “Where’s the ‘Strike’ in CrowdStrike,” Carr noted that, as of September 2012, CrowdStrike had announced the recruitment a number of highly skilled employees, but “so far they haven’t announced much in the way of a product line.” Carr was on record as stating that the Shady Rat report authored by Alperovitch was “an indictment of McAfee as an information security company,” noting that “it’s a lot easier to blame China than to acknowledge how you and your company have been profiting from a failed security model for all these years while hiding that fact from your customers.” With CrowdStrike, Carr observed, Alperovitch seemed to be repeating the same pattern of overselling his company’s capabilities.
“The company (CrowdStrike) website,” Carr writes, “claims to offer ‘Enterprise Adversary Assessment’ where ‘we identify the adversary and find out what they’re after.’ And how do they do that? Back to the website: ‘Through hunting operations, including host-based detection, threat-specific network analysis, and victim threat profiling.’” Carr, however, is critical of these claims, noting “CrowdStrike cannot currently deliver anything unique in the infosec space…that other companies aren’t already doing unless it significantly improves its sources and methods regarding identifying adversary state and non-state actors and pushes the envelope on active defense.”
The biggest sin, according to Carr, was the fact that the CrowdStrike methodology represented little more than “a continuation of the piss-poor intelligence that Dmitri Alperovich published while at McAfee,” singling out the Shady Rat paper as a case study in point. “There’s over 30 nation states developing computer network attack, defense, and exploitation capabilities,” Carr notes, “and at least a dozen that are highly proficient and actively conducting cyber espionage yet somehow McAfee’s ‘intelligence analysts’ only see China.”
Carr points out that while CrowdStrike “talks about identifying adversaries via toolmarks and the usual TTPs 9tools, techniques and procedures) that every so-called cyber intelligence firm narrowly focuses their attention on but that’s not analysis…that’s a cognitive trap known as target fixation. If after looking at all of the technical parameters,” Carr concludes, “the only nation state that you see is China, you need to find another job because you suck as an intelligence analyst.”
A former intelligence analyst named Michael Tanji echoes Carr’s concerns. Tanji spent 20 years working for the Defense Intelligence Agency, the National Security Agency, and other intelligence organizations, where he specialized in computer network operations and computer forensics. In an article entitled, “Malware Analysis: The Danger of Connecting the Dots”, Tanji asked the following question: “If I give you a malware binary to reverse engineer, what do you see?” His answer is telling: “Exactly what the author wants you to see.”
I want you to see words in a language that would throw suspicion on someone else. I want you to see that my code was compiled in a particular foreign language (even though I only read and/or write in a totally different language). I want you to see certain comments or coding styles that are the same or similar to someone else’s (because I reuse other people’s code). I want you to see data about compilation date/time, PDB file path, etc., which could lead you to draw erroneous conclusions have no bearing on malware behavior or capability.
Extrapolating from Tanji’s words, one sees that, when it came to Shady Rat, Alperovitch wanted to see China, so he did. This was the ultimate flaw in the methodology Alperovitch brought with him from McAfee to CrowdStrike, a willingness to make assumptions based upon misplaced certainty, to shoehorn unknowns into these assumptions, to allow personal bias to dictate the data set, and to let personal animus influence conclusions that might not otherwise be valid. This is what Alperovitch did with Shady Rat while working for McAfee back on 2011. Five years later, history repeated itself: CrowdStrike, with Alperovitch in the lead, fell into the exact same target fixation cognitive trap when it came to Cozy Bear and Fancy Bear.
Part Three: Guccifer 2.0
Alperovitch and CrowdStrike nearly pulled it off. The “name and shame” strategy is designed to embarrass state-sponsored actors, compelling them to cease and desist their criminal cyber activity while mobilizing political support at home for more robust cyber security policies intended to keep the identified perpetrators at bay. Had the attack on the DNC server been an actual Russian state sponsored event, this approach might have worked.
Almost immediately after the one-two punch of the Washington Post article/CrowdStrike technical report went public, however, something totally unexpected happened — someone came forward and took full responsibility for the DNC cyber attack. Moreover, this entity — operating under the persona Guccifer 2.0 (ostensibly named after the original Guccifer, a Romanian hacker who stole the emails of a number of high-profile celebrities and who was arrested in 2014 and sentenced to 4 ½ years of prison in May 2016) — did something no state actor has ever done before, publishing documents stolen from the DNC server as proof of his claims.
Hi. This is Guccifer 2.0 and this is me who hacked Democratic National Committee.
With that simple email, sent to the on-line news magazine, The Smoking Gun, Guccifer 2.0 stole the limelight away from Alperovitch. Over the course of the next few days, through a series of emails, online posts and interviews, Guccifer 2.0 openly mocked CrowdStrike and its Russian attribution. Guccifer 2.0 released a number of documents, including a massive 200-plus-missive containing opposition research on Donald Trump.
Guccifer 2.0 also directly contradicted the efforts on the part of the DNC to minimize the extent of the hacking, releasing the very donor lists the DNC specifically stated had not been stolen. More chilling, Guccifer 2.0 claimed to be in possession of “about 100 Gb of data” which had been passed on to the online publisher, Wikileaks, who “will publish them soon.”
With the foundational premise of his attribution report under attack, Dmitri Alperovitch responded, updating his “Bears in the Midst” report with the following passage (which was also sent out to the media as a press release):
CrowdStrike stands fully by its analysis and findings identifying two separate Russian intelligence-affiliated adversaries present in the DNC network in May 2016. On June 15, 2016 a blog post to a Wordpress site authored by an individual using the moniker Guccifer 2.0 claimed credit for breaching the Democratic National Committee. This blog post presents documents alleged to have originated from the DNC.
Whether or not this posting is part of a Russian Intelligence disinformation campaign, we are exploring the documents’ authenticity and origin. Regardless, these claims do nothing to lessen our findings relating to the Russian government’s involvement, portions of which we have documented for the public and the greater security community.
Through this release, Alperovitch and CrowdStrike sought to deflect the impact of the Guccifer 2.0 bombshell by treating it simply as an extension of the “Russia did it” narrative they had begun with “Bears in the Midst” — in this case, as a disinformation campaign. To do this, however, CrowdStrike would need to dissect claims made by Guccifer 2.0 regarding his hack of the DNC server. To accomplish this, CrowdStrike was assisted by a number of private cyber security companies, who began dissecting the Guccifer 2.0 narrative with an eye toward disproving his claims. The results have been less than convincing.
On of Guccifer 2.0’s claims is that he hacked the DNC using what is called a “zero-day exploit” of the NGP VAN software. He then installed a “shell code” into the DNC server, which he used to gain access to the entire DNC network. Guccifer 2.0 claims he used the DNC’s Windows-based domain architecture to infect several DNC computers with “Trojans”, and that he moved from one PC to another every week (i.e., “lateral movement”) to avoid detection, defeating intrusion detection software through the use of what he called “heuristic algorithms.”
ThreatConnect, a respected collective of cyber security experts, has been dismissive of Guccifer 2.0’s “zero-day” claim. Their argument, however, is circular and non-persuasive. They note that the MITRE common vulnerabilities and exploitation (CVE) website does not list any known vulnerabilities for NGP VAN software. A “zero-day” vulnerability is, by definition, an unknown exploit that would not appear in a CVE database until which time it had been identified and a “patch,” or fix, implemented.
ThreatConnect likewise sought to conflate Guccifer 2.0’s claim to have penetrated the DNC server in the summer of 2015 with the only publicly known vulnerability in NGP VAN — the December 2015 VoteBuilder breach that was audited by CrowdStrike, creating the impression that Guccifer 2.0 was claiming to have exploited a vulnerability that had not yet come into existence. But Guccifer 2.0 made no such claim. He simply said he penetrated the DNC server in the summer of 2015 using a “zero-day” vulnerability in the NGP VAN software.
CrowdStrike had stated in its various reports that the DNC server was, in fact, penetrated sometime in the summer of 2015. It also noted that it had uncovered no evidence on how this penetration was accomplished. While CrowdStrike and others speculate that the DNC fell victim to a “phishing” attack, the fact remains that Guccifer 2.0’s claim to have used a “zero-day” vulnerability remains uncontradicted by any evidence.
ThreatConnect also contended that if Guccifer 2.0 had, in fact, penetrated the DNC server in the summer of 2015, this penetration should have been detected as part of the DNC/NGP VAN/CrowdStrike audit. Missing from this logic is the fact the all parties acknowledge that the Cozy Bear actor had penetrated the DNC server in the summer of 2015, and yet this actor had somehow avoided detection by the aforementioned audit. Moreover, the tools and techniques Guccifer 2.0 claims to have used against the DNC closely mirror those used by Cozy Bear that were uncovered by CrowdStrike.
The confluence of time of access and tools and technique between Guccifer 2.0 and Cozy Bear begs the question — could Guccifer 2.0 be Cozy Bear? This was not a possibility considered by CrowdStrike, or any of the cyber security vendors who have commented on Guccifer 2.0. Instead, they have painted Guccifer 2.0 as an extension of Fancy Bear, citing forensic evidence that links Fancy Bear to documents stolen from the DCCC and subsequently released by Guccifer 2.0.
Guccifer 2.0 was always treated as either a third intruder, or else an extension of Fancy Bear. No evidence has been uncovered suggesting the presence of a third intruder. As such, Guccifer 2.0’s claims of having accessed the DNC server through a “zero day” vulnerability sometime in the summer of 2015 have been dismissed out of hand, leaving only one possibility — Guccifer 2.0 was Fancy Bear. Since CrowdStrike had attributed Fancy Bear as being Russian military intelligence (the GRU), by extension Guccifer 2.0 was the GRU.
Such a finding was very convenient for CrowdStrike, since it did not alter the conclusion reached in the June 2016 “Bears in the Midst” report. It also highlighted the tunnel vision CrowdStrike and the other cyber security companies had when it came to looking at the data emerging about the DNC cyber attack.
“If I give you a malware binary to reverse engineer, what do you see?” This is the question that had been posited by Michael Tanji, the retired cyber intelligence analyst. “Exactly what the author wants you to see.”
I want you to see words in a language that would throw suspicion on someone else.
An article published in ArsTechnica highlighted the work of an independent security researcher, Adam Carter, who had uncovered evidence that some of the documents released by Guccifer 2.0 in his initial document dump had been manipulated in a manner which introduced Russian words, in the Cyrillic alphabet, into the metadata of the documents, including a reference to “Felix Edmundovich,” the first name and patronymic of the founder of the Soviet security service, Felix Dzerzhinsky. The combination of the Cyrillic alphabet and the reference to a Russian spymaster seems ideal if one is trying to attribute its existence to the Russian intelligence services.
I want you to see that my code was compiled in a particular foreign language (even though I only read and/or write in a totally different language).
FireEye, a well-known cyber security company, has written a report on APT-28 (another name for Fancy Bear), highlights a number of Russian language indicators, including the consistent use of Russian language in malware code over the course of six years.
I want you to see certain comments or coding styles that are the same or similar to someone else’s (because I reuse other people’s code.)
Fidelis Security, another well-known cyber security company, was provided samples of the Cozy Bear and Fancy Bear malware for “independent analysis.” According to Fidelis, these samples matched the description provided by CrowdStrike and “contained complex coding structures and utilized obfuscation techniques that we have seen advanced adversaries utilize in other investigations we have conducted,” Michael Buratowski, the senior vice president of security consulting services at Fidelis, noted. The malware was “at times identical to” malware that other cyber security vendors, such as Palo Alto Networks, have attributed to Fancy Bear. Many of these similarities have been previously identified by other cyber security vendors and made public as far back as 2013.
I want you to see data about compilation date/time, PDB file path, etc., which could lead you to draw erroneous conclusions have no bearing on malware behavior or capability.
FireEye, in its report on APT-28 (i.e., Fancy Bear), also notes that the compile times associated with the malware align with the work hours and holiday schedules of someone residing in the same time zone as Moscow and St. Petersburg.
The fascinating thing about Michael Tanji’s observations was that they were made in 2012, largely in response to the spate of China attributions headed up by Dmitri Alperovitch’s highly publicized 2011 Shady Rat report. Four years later, the fixation on pattern-derived attribution remained a problem within the cyber security collective, this time with Russia as the target de jour. In 2011, the Chinese caseload was spread across a broad field of separate cyber attacks. In 2016, the Russian data set was limited to a single event — the DNC cyber attack.
Moreover, the data set in 2016 was under the exclusive control of a single entity — CrowdStrike. While select malware samples were farmed out to like-minded vendors, for the most part outside analysis of the DNC cyber penetration was limited to the information provided by CrowdStrike in its initial report. Even the FBI found itself in the awkward position of being denied direct access to the DNC servers, having instead to make use of “forensic images” of the server provided by CrowdStrike, along with its investigative report and findings.
There is much unknown about these scans — were they taken from May 6, when CrowdStrike first detected what it assessed to be a Russian presence inside the DNC server? Or are they from June 10, the last day the server was in operation? The difference could be significant, keeping in mind the fact that there were more than 30 days between the two events.
In this intervening time, CrowdStrike watched Guccifer 2.0 exfiltrate documents. It also possibly engaged in offensive measures, such as the dangling of so-called “attractive data” (the Russian-language tainted opposition research documents come to mind.) The possibility of additional manipulation of data cannot be discounted. However, even though members of Congress are starting to call for the FBI to take physical possession of the server and conduct its own independent forensic investigation, the server remains in the possession of the DNC.
Through the release of its “Bears in the Midst” report, CrowdStrike anticipated that the US government and, by extension, the American people, would place their trust in CrowdStrike’s integrity regarding Russian attribution. The media has, for the most part, accepted at face value CrowdStrike’s Russian attribution regarding the DNC cyber attack.
The US government, while slower to come onboard, eventually published a Joint Statement by the Office of Director of National Intelligence and the Department of Homeland Security in October 2016 that declared, “The recent disclosures of alleged hacked e-mails…by the Guccifer 2.0 online persona are consistent with the methods and motivations of Russian-directed efforts.”
On December 29, 2016, the FBI and DHS released a Joint Analysis Report (JAR) that directly attributed the presence of both the Cozy Bear and Fancy Bear actors on the DNC server to “spearfishing” attacks, thereby eliminating from consideration any possibility that Guccifer 2.0 penetrated the DNC server through a “zero day” exploit. This was a curious assessment, given that the only data in existence regarding what had transpired inside the DNC server was the data collected by CrowdStrike — data CrowdStrike maintains did not provide evidence pertaining to how the DNC server was initially breached by either Cozy Bear or Fancy Bear.
The Director of National Intelligence followed up with a National Intelligence Assessment, released on January 6, 2017, that similarly endorsed the findings of CrowdStrike when it came to Russian attribution for the Cozy Bear and Fancy Bear penetration of the DNC, as well as linking Guccifer 2.0 to the GRU, or Russian military intelligence. It was the strength of this national assessment that closed the book on debate on the matter of Russian attribution. Senators and Congressmen, intelligence officials and media pundits — all seem to be in agreement that Russia was singularly responsible for the cyber attack against the DNC, and the subsequent release of documents acquired from that breach. “Without a doubt,” “undeniable,” “incontrovertible” — this was the verbiage that accompanied any discussion of the case against Russia.
The genesis moment for this collective clarity, however, remains the carefully choreographed release of the CrowdStrike report, “Bears in the Midst,” and the accompanying Washington Post exclusive laying the blame for the DNC cyber attack squarely at the feet of Russia. From this act all else followed, leading to the certainty that accompanied this attribution was enough to overcome the challenge posed by the sudden appearance of Guccifer 2.0, enabling the same sort of shoehorned analysis to occur that turned Guccifer 2.0 into a Russian agent as well.
Much of this discussion turns on the level of credibility given to the analysis used by CrowdStrike to underpin its conclusions. Alperovitch, the author of the “Bears in the Midst” report, does not have a good record in this regard; one need only look at the controversy surrounding the report he wrote on Shady Rat while working for McAfee. A new report released by Alperovitch and CrowdStrike casts further aspersions on Alperovitch’s prowess as a cyber analyst, and CrowdStrike’s overall methodology used to make its Russian attribution.
On December 22, 2016, CrowdStrike published a new report purporting to detail a new cyber intrusion by the Fancy Bear actor, titled “Danger Close: Fancy Bear Tracking of Ukrainian Field Artillery Units.” This analysis, prepared by Adam Meyers, CrowdStrike’s vice president for intelligence, was claimed to further support “CrowdStrike’s previous assessments that Fancy Bear is likely affiliated with the Russian military intelligence (GRU). This report was used to promote a Jan. 4 live discussion event with Meyers and Alperovitch, titled “Bear Hunting: History and Attribution of Russian Intelligence Operations,” which was intended to educate the audience on the links between Fancy Bear and the GRU.
The “Danger Close” report was presented as further validation of CrowdStrike’s Falcon Program, which CrowdStrike claims helps organizations stop cyber penetrations through proactive measures developed through a deep understanding of the adversary and the measures needed to stop them. It was Falcon that “lit up” ten seconds after being installed on the DNC server back on May 6, 2016, fingering Cozy Bear and Fancy Bear as the culprits in the DNC attack. Falcon was now being linked to this newest effort at Russian attribution.
The only problem for Meyers, Alperovitch and CrowdStrike was that “Danger Close” was wrong — dead wrong — in every aspect of its analysis. The report was dissected by none other than Jeffrey Carr — the same individual who criticized Alperovitch over his Shady Rat claims. One of Carr’s most important findings deals directly with the credibility methodology used by CrowdStrike to attribute Fancy Bear to the GRU. “Part of the evidence supporting Russian government involvement in the DNC and related hacks (including the German Bundestag and France’s TV5 Monde),” Carr writes, “stemmed from the assumption that X-Agent malware was exclusively developed and used by Fancy Bear. We now know that’s false, and that the source code has been obtained by others outside of Russia.” Carr cites at least two examples, one a security company, the other a hacker collective, of the X-Agent malware existing “in the wild.” If these two entities have the X-Agent malware, Carr notes, “then so do others, and attribution to APT28/Fancy Bear/GRU based solely upon the presumption of ‘exclusive use’ must be thrown out.”
In one fell swoop, Carr destroyed the very premise upon which CrowdStrike not only attributed the DNC cyber attack to Russia, but the heart and soul of CrowdStrike’s business platform — the Falcon Platform used by CrowdStrike to provide “end point” protection to its clients. Far from representing an intelligent platform capable of discerning threats through advanced algorithms and proprietary techniques, the Falcon Platform seems to be little more than a database pre-programmed to deliver a preordained finding — X-Agent equals Fancy Bear, and Fancy Bear equals Russia.
X-Agent was always the one malware CrowdStrike could turn to as demonstrating an exclusive Russian attribution — every other malware detected in the DNC penetration was publicly available. Now it appears that X-Agent, too, was “in the wild,” available to any enterprising hacker to use as he or she saw fit.
Carr’s findings do not exclude Russia as a suspect in the DNC breach. It just means that Russia is not the only actor capable of using that particular malware — Alperovitch’s “DNA” is no longer conclusive. And if the presence of X-Agent no longer automatically equates Fancy Bear with Russia, then the same can be said about Guccifer 2.0. In short, the entire foundational premise upon which CrowdStrike, and by extension the US intelligence community, constructed its case for Russian attribution, just falls apart.
Within elements of the cybercommunity, the credibility of CrowdStrike has been shattered by its involvement in the DNC hack. Its two premier product paltforms — Falcon and Overwatch — have been exposed as being fundamentally (and perhaps fatally) flawed. The attributions derived from Falcon are little more than false positives generated by algorithms pre-programmed to deliver an outcome — CrowdStrike was looking for Russia, and therefore found it.
Moreover, the performance of CrowdStrike’s other premier product, Overwatch, in the DNC breach leaves much to be desired. Was CrowdStrike aware that the hackers continued to exfiltrate data (some of which ultimately proved to be the undoing of the DNC Chairwoman, Debbie Wasserman Schultz, and the entire DNC staff) throughout the month of May 2016, while Overwatch was engaged? Did Overwatch detect the spread of malware into the servers of the DCCC? If the answer is yes, one must question the competence of a cyber security company whose job is to prevent just that kind of activity. Did Overwatch help disseminate documents through legally questionable techniques designed to track an adversary’s activities? If so, the success of Guccifer 2.0, and ultimately Wikileaks, in publishing the stolen material undercuts any argument in favor of that exercise.
For more than 30 days CrowdStrike had exclusive control over the DNC server. During this time, CrowdStrike made an attribution of Russian involvement that has been shown to be fundamentally flawed, and oversaw the transfer of politically sensitive documents, some of which may have been tainted by CrowdStrike’s own actions, to parties who subsequently leaked this material in a politically impactful manner. In short, this 30-plus day period of time emerges as one of the most critical moments in the entire Russia election meddling saga. The data that would shed important light into the most significant claims of Russian involvement in the 2016 US Presidential election — the “weaponizing” of stolen documents — remains out of the hands of those conducting the investigation.
It is difficult to avoid the conclusion that CrowdStrike, by irresponsibly and deliberately attributing the DNC cyber attack to Russia, is involved in deception on a scale that is hard to measure. America today is virtually paralyzed as a result, with a new Cold War looming in terms of US-Russian relations, a possible Constitutional crisis brewing between the president and Congress, and the potential of a trade war between Europe and the US over sanctions passed by Congress in large part because of the issue of Russian election meddling.
Maybe Russia did it. This conclusion cannot be discounted. But such a finding can only be had after a thorough investigation of all available data that takes into account all possibilities. The intelligence underpinning the US government’s case against Russia has been undermined by the flaws that have been exposed in the analysis and methodologies used by CrowdStrike to make the Russian attribution the FBI and US intelligence community is using as its starting point. If, for instance, one changed “Russia” to “private hacker collective,” the entire premise of Russian involvement, and with it Russian collusion, is undone.
This doesn’t mean the Russian government didn’t have a favorite horse in the US Presidential race, or that Russian media outlets didn’t take an editorial stance in favor of a given candidate. It does, however, significantly weaken the foundational arguments surrounding Russian meddling that were built around the hacking of the DNC server, and the subsequent release of documents.
Given the stakes involved, one would think it would become a top priority of the US government to take control of the DNC server and conduct a thorough forensic examination of all activity, with special attention paid to the period between May 5, 2016 — when CrowdStrike installed its Falcon software and deployed its Overwatch capabilities — and June 12, 2016, when CrowdStrike purged the DNC server of all malware. There is every reason to believe that, in doing so, investigators will expose one of the greatest cons in modern American history.
Note: The author contacted CrowdStrike for comment about the employment of Falcon Overwatch as part of CrowdStrike’s 2016 DNC incident response. CrowdStrike responded with an email noting that they had no “no new insights or context to share about the investigation,” and directed the author to its June 14, 2016 report “Bears in the Midst,” as well as a March 5, 2017 article in WIRED. Both sources were consulted by the author in the research for this article prior to his request for comment. The author also reached out to Mr. Steven Chabinsky, the former CrowdStrike legal counsel, and Mr. Michael Sussman, the DNC legal counsel, for comment on various aspects of this article. Neither had responded by the time this article was published.
Scott Ritter is a former Marine Corps intelligence officer who served in the former Soviet Union implementing arms control treaties in the Persian Gulf during Operation Desert Storm, and in Iraq overseeing the disarmament of WMD. He is the author of Deal of the Century: How Iran Blocked the West’s Road to War (Clarity Press, 2017).