Cognitive Strategy Prompts for Human Centered AI Opportunity Detection
This article is based on the publication “Cognitive Strategy Prompts” (Creativity & Cognition 2022, ACM), co-written with , and Mary Mikhail.
When it comes to innovating a service or business, it seems like “AI” is the thing to do in 2023. However, instead of further defining the problem, we find our stakeholders play AI buzzword bingo by throwing the latest and greatest trending “AI terms” into the conversation.
Even worse, our client might expect us to “just apply some AI Magic” to solve any pain point and bottleneck that might occur in their business.
So, how might we identify precise, actionable and human-centered problems that are worth solving with AI methods? How can we adapt user research activities and design process to facilitate opportunity detection for AI powered innovation?
Cognitive Strategies
There is plenty of terminology to describe AI systems and features.
However, we have much less words to describe and decipher the processes and kinds of knowledge work we want to assist.
Knowledge workers Search for, Investigate, and Make decisions based on information on a daily basis. Based on the categorisation of information retrieval needs and cognitive processes, we structure information seeking, obtaining and handling of information into 5 different categories and prompts: Learn, Lookup, Relate, Monitor & Extract, and Create.
Each category has its unique cognitive strategies and might pose different challenges for the application of data science methods. For example to differentiate, contrast or relate data points when investigating information (see paper for full list of prompts).
Application
After all, AI is defined as a methodology that ‘aims to mimic the problem-solving and decision-making capabilities of the mind’. The proposed methodology of cognitive strategy prompts gives us a clear and actionable linguistic platform to link the human cognitive processes to the corresponding AI methodologies.
Prompts might support initial discovery and user research, serve as prompts during workshops, or facilitate user needs mapping and reframing of data science problems.
Cognitive Strategy Prompts for user research
The prompts aim to support user research, e.g. creation of moderation guides, probing during interviews, as well as analysis such as process mapping, jobs-to-be-done or task analysis.
Let’s look at an example.
Let’s imagine that we want to research how travellers plan and book holidays. We might use the prompts to investigate whether travellers are “trying to get an overview” on travel options, or whether they “search” for specific holiday destinations.
We can also apply prompts in participatory co-creation workshops. During our experimental sessions, we conducted a jobs writing exercise, in which we asked participants to write jobs-to-be-done for a given scenario, e.g. activities that are involved in the “travel planning”.
After an initial understanding of process and tasks was established, we introduced the prompts. Participants were asked to think again, which “cognitive strategies” they might apply at which stage and refine the jobs accordingly. The refined jobs statements were used for further problem definition and ideation.
Reframe Prompts into Data Science Problems
“When you have a hammer everything looks like a nail”. It can be challenging to translate conceptual ideas into data science problems. Often, stakeholders might “lock into known AI buzzwords” (e.g. “I heard about deep learning, can we solve XYZ with deep learning?”), scientists might be biased by methods they recently used on other projects.
In a survey we asked data scientists to associate each prompt to a number of data science and AI methods.
The resulting mapping aims to provide options and help translate between scientists and non-technical stakeholders.
When reviewing job stories, a data scientist might use the mapping to generate alternative solutions that might support the user need and underlying cognitive strategy. In reverse, they might also think through what cognitive strategies a data science method might enable.
Another example:
When a news reporter is trying “to get an overview” on current events and news updates, she might well benefit from a system that applies methods such as clustering, anomaly detection or topic modelling. Ideally, this might mimic and support her approach to identify themes and relationships within news events. Different experiments with available data sets can explore which direction might be most promising for the use case.
Considerations
Initial experiments have been overwhelmingly positive. We are aiming to expand on the cognitive strategy prompts considering the various nuances within industries.
The cards have been developed in the context of knowledge work and research for B2B services. Cognitive strategies might be different for different industries and domains, as well as for consumer products and services.
We experienced that its beneficial to work with a (very) limited set of prompts. Even the set of 10 cards proved to be challenging during some of our workshops.
More detail in the paper “Cognitive Strategy Prompts” (Creativity & Cognition 2022, ACM).
We are keen to improve the framework! Let us know your thoughts!
