Intelligence Analysis Basics — How Data Becomes Actionable Insights
I recently explored some of the basics of the U.S. intelligence community, its agencies, and how they arrive at conclusions on behalf of decision makers. Because of the sheer volume of credible allegations of espionage on behalf of the President-Elect of the United States, much of the news will involve information released by intelligence agencies. It is important for Americans and citizens of the world to get minimally aware of how these institutions operate on a daily basis to provide security.
The last article dealt with the basic ideas of gathering sources and corroborating them — within and across different agencies — to arrive a picture of the world on which leaders can make big decisions. To expand on this, here is more of the nitty-gritty reality of wrangling disparate sources of data of varying quality, completeness, and reliability, and how this hopefully reaches sufficient certitude that we make the right decisions for individuals, institutions, and the world.
Intelligence Analysis: A hierarchy founded on data, building toward value
The principle concept to understand in intelligence analysis is that analysts are trying to understand a world they cannot see by piecing together a mass of data from sources that can vary wildly in reliability, completeness, and bias. As I noted in the last article, each of our national intelligence agencies specializes in certain types of data.
The Central Intelligence Agency collects human intelligence (HUMINT) that essentially means a whole lot of stories from people.
The National Geospatial-Intelligence Agency are the folks with all the spy satellites, collecting an unfathomable volume of images and measurements, from pictures of the Earth to gravity maps, all over every millimeter of the globe.
The National Security Agency is stockpiling massive troves of radio signals, phone calls, emails, fax transmissions, and communications between machines.
In addition to the institutions themselves, each of them has employees who, individually and collectively, possess data. They remember facts, practices, assumptions, memories, and these too are assets, albeit ones that are hard to quantify and access in a predictable manner.
Any of these individual pieces of data collected is useless. It’s impossible to impart just how much raw data each of these organizations possess, and how the information itself has no value. Data only has value when human analysts cross-check data for quality and reliability and build it toward a story that changes the minds of decision makers about what actions they should take to create value in line for their core missions.
There are many models of how to organize data for maximum effectiveness as an intelligence product, but my favorite remains the Knowledge Value Chain, from my dear friend and colleague Tim Powell. There are many similar, but none as elegant in its description of how a bunch of meaningless numbers and words become a decision that really matters.
It all starts with the acquisition of data. You pull in numbers off a computer or emails from some source trying to tell you something. It’s a hard drive of God-knows-what. This unsorted mass of data has been exploding in volume exponentially, while our tools to handle them lag a bit, and people grow way less than exponentially in their ability to process this data.
When either people or software tools interpret this first layer, it become information. It goes from meaningless characters to, “Wow, the German ambassador to France said X.” or “Temperature in the Gulf of Mexico is 0.2 C hotter than normal.” None of this actually matters to a decision yet, it’s just more refined than raw data.
If you organize a significant amount of information, you arrive at knowledge. Let’s say — for only one crazy example — you have information that Lt. Gen. Mike Flynn, the presumed National Security Advisor for the incoming Trump Administration made five phone calls to the Russian Ambassador on the day Obama announced new sanctions. Hypothetically, the NSA has the raw intercepts, an analyst listened to them (SIGINT), and a spy in the Russian Embassy snuck out a confirmation (HUMINT). Now you’ve arrived at knowledge that prior to his assumption of duties, one of Trump’s team is in contact with Russians.
Combine this with a much bigger part of the story, for example the connections between Paul Manafort and a pro-Putin candidate, and the strange connections between pro-Russia advisor Carter Page and the Kremlin, and now you’re shaping up to an intelligence assessment. Data in its raw format is becoming increasingly refined by intelligence analysts who come together to see a picture form.
Egads, it seems like there are rather unnatural ties between the incoming Administration and a hostile foreign power!
Now you get to the top of the pyramid. You make a decision to take an action that you hope will produce the value you are looking for. There are some realities here: you will never have enough data, information, knowledge, and intelligence to support the most important decisions. You will always want more. And yet time waits for no one, and at some point a leader relies on his or her judgment to do the right thing.
This process has been happening in real-time — in overdrive, really — since America has arrived at the historical crisis point of recognizing that a foreign power has meddled in our elections and that their chosen candidate wishes to destroy the NATO alliance and cozy up to that same foreign power.
I just pray that our leaders have the intelligence they need to provide maximum value — to save our country from grave harm.
Originally published at www.ericgarland.co
