Activity-Based Intelligence: Understanding Patterns-of-Life
By Patrick Biltgen, Ph.D.; Todd S. Bacastow, Ph.D.; Thom Kaye; and Jeffrey M. Young
This article was originally published in USGIF’s State & Future of GEOINT Report 2017. Download the full report here.
Activity-based intelligence (ABI) is an analysis methodology that rapidly integrates data from multiple sources to discover relevant patterns, determine and identify change, and characterize those patterns to drive collection and create decision advantage. Unlike the traditional intelligence cycle, which decomposes multidisciplinary collection requirements from a description of the target signature or behavior, ABI practitioners have advanced the concept of large-scale data filtering of events, entities, and transactions to develop understanding through spatial and temporal correlation across multiple data sets.
Since the Babylonians created the first geospatial intelligence (GEOINT) products around the 5th century B.C., time and geography were essentially decoupled. Cartography represented the world at a snapshot in time. When the first photoreconnaissance satellites were lofted in the 1960s, the focus was largely on locating large, fixed military installations and mapping Soviet territory. Periodic sampling on the order of days or weeks was sufficient for arms control treaty monitoring of hostile foreign nations. Events such as the Soviet Invasion of Czechoslovakia, the Great Scud Hunt of Operation Desert Storm, and the search for terrorists post-9/11 drove demand for sources and methods of GEOINT collection and analysis that could capture dynamic activities. The focus shifted from describing large-scale activity to fine-grained “patterns-of-life” of individual entities.
A pattern-of-life is “the specific set of behaviors and movements associated with a particular entity over a given period of time.”[1] The focus on the individual is the fundamental uniqueness of the ABI method and drives the need for a new set of techniques and approaches to intelligence analysis. Technological advancements of the past two decades — a revolution in information technology and the dawn of “big data” — enhance our ability to collect and process large volumes of data. Tradecraft advancements including both mind-set shifts and new analysis methods allow analysts to make sense of this flood of data to understand individual behaviors and activities in the context of the environment. By resolving entities and understanding patterns-of-life, analysts can build models of potential outcomes and anticipate what may happen.
Technological Advancements
Inferring an outcome from a small number of observations is a dangerous proposition, especially when the dynamic and often unpredictable actions of humans are concerned. ABI promotes a deductive approach to analytic reasoning. Deduction reduces the space of potential outcomes by eliminating the impossible, but because much more data is required, technological advancements that improve the resolution and ubiquity across the spatial, temporal, and spectral dimension are needed.
Advancements in spatial resolution and ubiquity are the most intuitive for GEOINT professionals. Since the early days of GEOINT, we have seen increasingly crisp and colorful imagery over increasingly large swaths of the Earth. Start-ups such as Planet promise to image our Earth daily, providing unprecedented insight into human activities at global scale. Transportation start-up Uber partnered with DigitalGlobe to use high-resolution imagery to identify new lanes and traffic patterns, pinpoint the optimal pickup and drop-off locations, and ultimately improve the passenger experience. Thousand-dollar commercial drones beam ultra-high-definition video to smartphone-based controllers. ABI benefits from improvements in the spatial dimension because individual activity patterns can be discerned from the background. This improves the accuracy of pattern-of-life analysis because analysts don’t have to infer what is happening inside or in between each pixel.
The most stunning transformation of GEOINT in the past 15 years involves the temporal dimension. Simply put, video killed the photography star. Multiple dash-mounted cameras captured an exploding meteor in Chelyabinsk, Russia, in 2013. A U.S. Coast Guard camera caught US Airways Flight 1549’s miraculous landing on the Hudson River. Footage from body-worn, traffic, and security cameras pepper 24-hour newsfeeds. Today, there are more than 100 Air Force Predator and Reaper drones aloft, sending high-resolution video to ground sites around the world. New wide-area persistent surveillance systems such as the Air Force’s Gorgon Stare pod-mounted sensor package are used to track people and vehicles in city-sized areas. Google’s Terra Bella (formerly Skybox Imaging) stunned the world in 2013 when it released the world’s first high-definition video from space. Human motion is one of the most powerful activity indicators and is central to the development of an entity’s pattern-of-life. As affordable and persistent high-resolution sensors proliferate worldwide, mapping patterns-of-life becomes feasible because analysts also do not have to infer what is happening between scheduled, separated collection opportunities. For the first time, analysts can map complete motion transactions at massive scale.
Improvements in spectral diversity — the proliferation of sensors that capture increasingly broad samples across the electromagnetic spectrum including combinations of non-traditional phenomenologies — represent the most transformative and revolutionary advancement shaping the GEOINT Community today and into the future. Orchestrated “multi-intelligence” collection was a rare and expensive prospect when sensors were limited, but as the GEOINT marketplace “darkens the skies” with more and different kinds of sensors, simultaneous collection on a single entity will become commonplace. Development of a pattern-of-life using a single sensor type is extremely difficult because the sensitivity of collection cannot be improved without also increasing the false alarm rate. The integration of multi-sensor, co-collected data allows the strengths of one data source to compensate for the weaknesses of another, enabling analysts to paint the complete picture of an entity’s pattern-of-life. This approach also makes it much more difficult for an adversary to practice denial and deception because he may be observed in many different ways that are difficult to mask simultaneously. Finally, let’s not forget the ultimate “new” multi-INT sensor: our mobile devices that drip digital data as we move simultaneously through geospace and cyberspace. The ability to collect and integrate these information sources with relevant geospatial context provides the basis for developing accurate and robust patterns-of-life.
Tradecraft Advancements
The three technological advancements — in spatial, temporal, and spectral resolution — enhance ABI collection and provide a mechanism to capture a complete pattern-of-life, but making sense of all this data in time to do something about it represents a fundamentally new challenge for analysts. Three axioms enable the mind-set shift necessary to develop, understand, and formalize patterns-of-life.
The first axiom may be called “Clapper’s Law.” In 2004, then-director of the National Imagery and Mapping Agency (NIMA) James R. Clapper said, “Everything and everybody has to be somewhere.” This simple and powerful principle is the basis of the ABI pillar of “Georeference to Discover.” Spatially indexing all data allows discovery of activities. Everything happens somewhere. Nothing can be in two places at once. Nothing can be nowhere. Because of these existential constraints, analysts can implement the powerful techniques of hypothesis testing and deductive reasoning: By eliminating all the places where the entity is not, the one remaining place is where the entity must be. Repeated application of this process across time produces a bona fide pattern-of-life unique to a single entity.
A second axiom, the concept of time-geography developed by Swedish geographer Torsten Hägerstrand, formalizes the concept of a pattern-of-life by describing a path taken through space-time: “Life paths become captured within a net of constraints, some of which are imposed by physiological and physical necessities, and some imposed by private and common decisions.” Hägerstrand describes capability constraints “which limit the activities of the individual because of his biological construction and/or the tools he can command,” such as the hours of sleep required per night or the maximum velocity of a vehicle. Coupling constraints define “where, when, and for how long the individual has to join other individuals, tools, and materials in order to produce, consume, and transact.” This includes the interaction of the entity with objects such as cars, office buildings, and smartphones. Authority constraints describe the degree to which access to certain regions or resources such as a locked vault, a crowded communications channel, or a particular seat at a movie theater are controlled at a given time. An authority constraint might even be a social or religious preference such as vegetarianism or a defined prayer hour. This constraint-based approach formalizes the mathematical relationships that define an entity’s pattern-of-life. When linked with Clapper’s Law, Hägerstrand’s method helps analysts deductively narrow the possibility space by eliminating large chunks of infeasible “somewhere.”
The third axiom, widely known by geographers, is called Tobler’s First Law of Geography: “Everything is related to everything else, but near things are more related than distant things.” Tobler’s Law is the basis for spatial autocorrelation. Because nearer things are more likely to be significant, it further focuses reasoning efforts and the deductive process on the spatial and temporal conditions that are most likely to be true. When combined with the first and second principles, Tobler’s Law completes the mind-set framework necessary to implement ABI. Analysts know everything must be somewhere, they know how to constrain the possibility space, and they know how to prioritize geospatial information based on proximity.
Putting It All Together
USGIF board member Jeff Jonas noted that to catch clever criminals, “one must either collect observations the adversary doesn’t know you have or be able to perform compute over your observations in a manner the adversary cannot fathom.” Increases in spatial, temporal, and spectral diversity ensure almost everything about our lives may be captured by one or more sensors and stored indefinitely. The mind-set shifts driven by ABI provide the basis for a framework to reason through these large volumes of data, understanding how humans move across and interact with the Earth. Formulating patterns-of-life and separating the signal from the noise requires human analytical thought, but also new tools and approaches that make it easier for analysts to filter data, quantify patterns, and test hypotheses.
Standard statistical methods, regression techniques, and models are almost always based on the assumption that the variables are independent. But people are not lifeless particles governed by Brownian motion or Kepler’s laws; we are complex entities whose activities are constrained and influenced by geography and other societal, relational, biographic, historic, and preferential constraints as outlined in the three axioms. For these reasons, human activities are not entirely random processes. Seemingly unrelated activities and behaviors cast as a spatiotemporal narrative expose the previously undiscoverable threads of motivation, purpose, and implication. Integrating and studying historical data that describes the activities of an entity across time and space improves an analyst’s understanding of that individual’s pattern-of-life. Adding the set of constraints and likely outcomes produces a model of what the analyst thinks will happen and a series of hypotheses that can be tested with real-world observations.
Modeling techniques provide promise, but these cannot be large-scale, aggregated, population-level models, kinematic models of object motion, or statistical demographic-based models of behavior, because they fail to capture the nuances of the individual based on his or her behaviors, activities, beliefs, and motivations. The GEOINT discipline increasingly includes statistical methods, but analysts must be trained in mathematical techniques to avoid detecting and reporting on spurious correlations. Automation might save time spent on routine tasks like searching for and reformatting data, but as former National Geospatial-Intelligence Agency (NGA) analyst Stephen Ryan notes, the “dumpster diving” gives analysts an intimacy and familiarity with data that improves their ability to analyze it.
There is no magic “ABI tool” that features a “find bad guy” button. ABI provides a series of techniques, a change in mind-set, and a methodological framework for data analysis that is becoming increasingly important in our increasingly complex world.
Conclusion
Bloomberg forecasts that by 2030 the world will host 40 megacities — cities with at least 10 million people — with increasing population densities. These megacities will be filled with self-driving cars, automated restaurants, tracking cameras, and always-on streaming connectivity to the world. Dozens of digital transactions (beginning with your toothbrush’s daily report and ending with your pillow’s temperature optimization) will be required to operate your gadgets, your job, and your body. Almost every object in the world will have an Internet Protocol (IP) address and be connected to everything else. And every one of these objects and transactions will create another record in your lifestream and define your insuppressible pattern-of-life.
In his book Connectography, author and futurist Parag Khanna describes an evolution of the world from physical and political geography to a functional geography that describes how humans use and interact with the world. As mobility, telecommunications, energy, finance, and the supply chain are increasingly integrated and as people freely move through geospace and cyberspace, methods for ABI and the principles outlined above will become increasingly important in understanding the world.
With all the focus on spinning globes, dancing dots, and shiny satellites, it’s easy to lose sight of the fact that intelligence is about avoiding strategic surprise. NGA Director Robert Cardillo, in his 2014 “Director’s Intent” document, challenged his workforce to view GEOINT through “the lens of consequence.” Consequence is about anticipating what may happen, why, and what the GEOINT Community can do about it. ABI techniques, tools, and tradecraft are critical to understanding patterns-of-life. Integrating the activities of humans upon the ubiquitous foundation of our physical, cultural, and functional geography and continuously updating our knowledge of these interacting forces represents the next fundamental shift in the state of GEOINT.
[1] Patrick Biltgen and Stephen Ryan, Activity-Based Intelligence: Principles and Applications. Boston: Artech House, 2016.
