Cyber: Ignore the Penetration Testers

They only know about one part of one part of cyber

There are a lot of people generating a lot of text about cyberwar, cyberconflict, cyberweapons, cyber everything. They are, for the most part, completely wrong. I firmly believe that a technical background, particularly a security background, is critical to understanding the fifth domain (you guessed it: cyber.) But technical chops are not enough, or even necessarily a prerequisite, to speaking with authority on the subject.

The fog of cyberwar is the war.

Penetration testers have this problem where they frequently can’t see past the end of their Kali USB. They establish the false equivalence of: “China hacked $X; I can hack $X; therefore, I am an APT and an APT is like me.” An APT is literally the instantiation of a nation state’s will. It is not a toolchain.

The Cyber

The important thing to remember about the fifth domain is that it is not actually about technology — it is about people. How people perform sensemaking and interpret reality is a matter of what their devices inform them.

People are the problem with cyber.

There are other important angles to cover as well: technical capabilities; analytic skills; agility; information; purpose; deterrence; attribution; geopolitics; espionage and counterespionage; intelligence, and counterintelligence. I will not be covering those. Instead, I want to explore why people with a purely technical❖ understanding of hacking are so frequently wrong about the nature of fifth domain conflict.

❖ Although generally not as horribly wrong as those with no technical background at all.

Let’s Build an APT!

One reason that penetration testers are not very good at understanding the complexities of cyber conflict is that they see only the tooling. As a geek myself, I have a similar infatuation with clever tools and hacks, but tools are not that important. To quote Boyd:

People. Ideas. Hardware. In that order.

The same priority holds true in cyber. A powerful APT is not defined by its tools, but by the people and ideas they have, and then the toolchain that they develop to accomplish their goals (generally set for them by someone else.)

For our thought experiment, let’s assume that a customer wants to know a secret. Someone somewhere is guarding this secret, but it is on a computer so it is within our reach, given resources…

This will be a very lean, very focused APT, with limited flexibility. So, let’s dive right in — how to create an APT.

The Lean APT

You can’t download your way to parity with Ft Meade — Mara

1. Operators: hackers, developers, operations, etc

The part everyone is familiar with in cyber is the hacking, the exploits, the implants and toolchains. These need to be researched, developed, maintained, and put to use. Additionally, they need to run somewhere on something. So there has to be support staff, the sys admins and dev ops to keep the computers up.

1. Operator: 1
2. Developers: 1
3. Sys Admin: 1

Right now, this is almost the bare minimum, but we can double up some of the roles. An operator can be a developer, and the sys admin can be a developer (and/or an operator.)

Our basic team is, combined, a minimum of 2–5 people. They need to be well compensated because these are highly technical roles and the competition for security people is fierce. This is a lean APT, so no room for dead wood, so salary plus benefits and bonuses, is probably at $250,000 a year per person (there’s a discount because they get to actually hack, rather than juggle spreadsheets and go to meetings.)

Cost: 4 pax @ $250,000

Frontline Support

Support is a less technical role — laundering money and registering shell accounts is not quite the same skill set as x86 assembly and Windows kernel heap layouts. However, it is critical that all those exploits and tools developed by the ops team have infrastructure to run on, and that it can’t be traced directly to the group. Again, lean APT: there only needs to be one person to create shell companies, launder money, and register accounts for everyone. The compensation is less attractive than for the technical people, but still, this is a job that requires skills, so let’s say $125,000.

Cost: 1 pax @ $125,000

Analysis Staff

The fat edge of the wedge. The ops team can develop a toolchain and get the APT into any target. They have great infrastructure and support from the frontline team, and they are now exfiltrating data. This is great, exactly the point of having this APT group in the first place. The ability to get data. Except, well, most targets have gigabytes and terabytes of data. Even legal discovery (the other type of exfiltration no one ever talks about) involves searching massive numbers of files and documents for relevant information.

This lean APT is going to need an analyst or ten. Again, keeping it lean, we’ll stick to just a few post grads and one senior analyst to direct them. Since they are post grads, they aren’t super cheap, but this isn’t exactly paying English PhDs minimum wage to work as baristas. The number of analysts will be the bottleneck in the ability of the APT to generate useful intelligence. The operators got in, and got data out, but until it has been processed and turned into intelligence product it is pretty much useless. Fortunately, the job market for post docs is weak, so they are cheaper hires.

Cost: 1 x sr analyst @ $120,000, 5 x analysts @ $75,000

UPDATE: As was pointed out on Twitter, this analysis team doesn’t include any translators. In reality, there would be a lot of translators. In this example, we will pretend that the team is fluent in the target language (although, of course, this further limits the APT’s flexibility.)

Analysis Support

The analysts will have to sift through data, cull out the good bits, develop documents, reference and index previous findings, produce models, and generally need some ability to process and query the data. This means tools, computers, and support staff to keep that infrastructure running. This lean APT is adding headcount at an alarming rate, so let’s keep the support staff very minimal and buy as much off the shelf software as possible.

There are a lot of software offerings, but we’ll use IBM’s i2 analysts notebook.

Cost: 1 x sys admin @ $80,000, software costs: $20,000 (minimum)

UPDATE: A friend pointed out that actually, given the volume of data that will be exfiltrated and the need to process and search it, the analysis support team will probably need to build a “mini Google.” And i2 analysts notebook is completely the wrong tool for the job (however, it was the only one for which I could find a price.)

So the real cost here probably includes 2–3 engineers for setting up Hadoop clusters, search interfaces, and input/output processing filters. That is highly technical work, so probably $200,000 each. And of course, now the Analysis Support is large enough that it might require another manager, at least a project manager, to keep on track. This section of the team is probably going to run to over $500,000 by itself.

Administration

This lean and mean APT that can hack any single target (of a particular type) plus analyse and produce intelligence product is now over 10 people. This requires management overhead, because there has to be coordination and direction. If we make the manager handle the analysis and the operations (not a great idea, totally different skillsets) we can can get away with just one. However, there it would be far better to have two managers.

Cost: 2 x managers @ $125,000

Infrastructure

The headcount has grown to about 15 people, all of whom will need to be put into a building somewhere with computers and internet and a coffee machine. They need electricity, climate control (heating or cooling), security personnel and equipment (CCTV, good locks, etc.) This supporting infrastructure for the employees and their computers has a cost, even if it is kept low. I won’t even try to estimate it, but you can imagine where this is going…

The Point Is…

Even a lean mean targeted APT — they can break into any [specific] target, exfiltrate data, analyse it and produce intelligence product — operating for a single year, is over $2,000,000. That’s very cheap, of course. But it doesn’t include the infrastructure costs, the 24x7 guard personnel, any margin for purchasing tools rather than developing them in house, churn (the staff will move to other shops and need to be replaced), plus the coffee budget. There are 15 people working at this APT to produce intelligence, and only a couple of them are actually hacking or writing exploits.

Ignore The Pen Testers

This is where penetration testers get it wrong. They are the hackers and (sometimes) the exploit writers. They never have to exfiltrate data, or dwell on networks for months, or burrow so deep they can never be extracted, or analyse the data collected. The experience of a pen tester is very narrowly focused on just breaking into systems, which is their end goal. For the APT crew, the break in is just one step towards the goal.

If your plan is: first thing we do, we hack! You’re doing it wrong.

The above describes a general purpose, if small, team. Individual hacks may not need all the roles — sloppy security could facilitate greater use of off-the-shelf tools, insider sources could slash the analysis workload, and if you’re just after some cash, you might be able to do away with the analysis entirely; just wire some money to a convenient casino or three. But this is the right ballpark for a sustainable, consistent team producing useful, actionable intelligence against a range of targets.

This is my problem with the typical infosec crew doing analysis on cyber. They see only the hacking, and as such they see it as a near-trivial capability available to anyone suitably motivated with some technical expertise. Real cyber is much more involved. You’re not going to get it from hackforums, and it’s not within reach of every jihadi with a grievance.