Decision Intelligence, the evolution of AI/ML
In March 2022 twelve forward-looking battle-tested practitioners from the related fields of Decision Analysis and Data Science spent a day brainstorming how both fields could benefit from each other. The day generated an agenda of five themes that forge a new approach to Decision Intelligence.
Why this Combination?
Decision Analysis and Data Science are complementary and both are in need of a tune-up: Data Science modeling needs to be anchored in decision making, and Decision Analysis needs to become data-driven. To label this synthesis we borrow the term “Decision Intelligence” — -bringing together the data-driven analytics of Data Science within the larger context of decision modeling. The need is to mature the practice of Data Science and to extend the evidence-based reach of Decision Analysis. Fundamental to both fields is the principle of rational choice; meaning taking actions that meet one’s goals. This is obvious, but difficult to achieve in practice and tends to get side-tracked with the current promotion around “AI.”
By engaging both communities, we are driving an initiative to create a new kind of practice that is anchored in the union of decision-focused consulting and takes advantage of the profusion of software and cloud-based computing to improve and automate decisions. In addition to exploiting Data Science’s predictive and statistical aspects, Decision Intelligence solves the business alignment problem by integrating predictive and value modeling aspects. By “decision” we mean any action, choice, or control that affects an outcome — a transaction, a prescription, an investment, a change in direction, or routing of information, which is valuable only if it affects a decision. We need to be able to “connect the dots” from the decision to the outcome, a problem that recently has been recognized as “misalignment” but has a long been understood in the field of Decision Analysis.
Our previous workshop brought together a group of senior practitioners from the Decision Analysis, Data Science, Statistics, and High-Tech communities. Its findings are summarized under five themes, detailed below. We intend to build on these themes to create a new field, taking advantage of the energy and momentum that came out of the day, by hosting a one-day virtual conference, where we invite broader participation as the next step in making Decision Intelligence real.
Five Themes for a New Synthesis
1. Decision Analysis and Data Science,
2. A common vocabulary,
3. Humans at the center,
4. A new practice, and
5. Integrating value and predictive modeling.
As is true of any practice of an applied science, Decision Intelligence draws on several related disciplines.
- Integration of best practices of Decision Analysis and Data Science
Decision Analysis and Data Science bring complementary methods to business problems.
Data science is “everywhere”, but it’s also “all over the place.” The common view of Data Science as simply starting with a table of data, then optimizing predictive accuracy is oversimplified and inadequate to meet real world business challenges. Conversely Decision Analysis, which has traditionally not been data driven, can work in data-rich areas, using the expertise Data Science brings in aligning data discovery to the problem at hand.
Both fields share origins in common principles in Bayesian methods, and from early work in Artificial Intelligence and Statistics that we can build upon.
2. The need for common vocabulary
The different fields that Decision Intelligence draws upon use different terms for the same concepts. Just take the term “decision,” which in Decision Analysis implies a commitment of resources — -an individual making a tangible change in the world. A statistician uses the same word for the threshold determined by a p-value. To confuse things further, an organizational analyst might speak of “deciding on one’s value.” Terminology gets even murkier once the topic turns to probability and uncertainty. Can we agree on what are “inference”, “prediction”, “learning”, “information”, “features”, “co-variates”, and so on?
A common vocabulary begs for a detailed explanation of this new synthesis — -a new teachable canon that can borrow from not only Machine Learning, Statistics and Decision Analysis, but from Bayesian methods — -the graphical methods that causal models draw upon: Influence (e.g., Decision) Diagram notation, and the tenants of Decision Quality.
3. Humans (back) at the center
Models are artifacts for humans to use, not agents in their own right. Ultimately any decision involves an individual. Either the model is making a recommendation to a person or taking an action in their stead. Commitment to taking an action requires both “head and heart.” It is an act of will that brings in emotional, organizational, and moral considerations. Maybe the purely rational choice is not best? Behavioral psychology, specifically Prospect Theory tells us how individuals’ perceptions play into poorly made choices. Similarly organizational theory of “satisficing” explains why organizations fail.
How best do we engage the user’s involvement? For instance, recent Design Thinking methods help in listening to the customer when formulating their problem. And a user’s acceptance of model results relies on having an explanation that can best be done by explaining the model in causal terms.
4. The practice of Decision Intelligence
How does the field translate into practice? What will the tools and techniques of the profession look like? How does this manifest itself as a business opportunity?
Any modelling project that adds value will change the business process to which it is applied. This is true at operational, product design or strategic direction levels. Decision models have a comprehensive scope, including longer term goals and tradeoffs. Consequently, how such analysis changes an organization — -how peoples’ jobs will change — -is central to the analysis.
To model the entire decision process means that the practitioner must be involved in the effort from the beginning, when the customer’s issues are first translated into a model formulation. As the name implies, the practice of formulation makes the decision explicit. The formulation would start modeling an initial decision on whether to pursue the project or not and then be expanded to focus on subsequent, operational choices.
5. Integrating value and predictive modeling
Combining value and predictive modeling raises novel algorithmic questions. A model, when fully formulated is an integration of decision, value, and uncertain — -that is predictive — -functions. Models that currently are optimized against an accuracy objective will have a different optimization objective when optimized against value. For example, ROC curve analysis is an illustrative case of using a prediction to make value tradeoffs.
Under some conditions the predictive model and value model can be factored so the components can be learnt separately and then combined. Or one may take a comprehensive approach as in Reinforcement Learning, which attempts to solve stochastic, dynamic optimization problems for a given value model, while learning the predictive model simultaneously by combining “exploration” with “exploitation.” But Reinforcement Learning is brittle and data-inefficient, and its success depends on running a huge number of online trials — a luxury that few domains can tolerate.
