Using qualitative research to structure an unbiased AI dataset
How the design and AI teams collaborated in our AIDA project
When it comes to a sensitive domain, good accuracy is not enough. This is the reason beyond our AIDA project journey. As introduced in this article, AIDA is a research project born inside Tangity that aims at helping people with Autism Spectrum Disorder (ASD) and their caregivers — such as parents, educators, therapists and so on — to communicate effectively.
This article describes our experience of a collaborative process among AI and design teams in researching for AIDA; in particular, it will focus on how we used qualitative research insights to design the AI dataset and model. In doing so, we will explain how we leveraged on qualitative insights to define necessary quantitative data for machine training through an ethical approach.
Our way
“Towards an “Ethics by Design” methodology for AI research projects” authors discuss the need for a methodology that enables ethical design research, involving a broader set of skills since the beginning of the project. AIDA project aims to find an application to these suggestions, by testing a process to identify and reduce ethical AI-related risks. Reducing risks and negative impacts was particularly important for the AIDA project because of its sensitive domain: a design error in the autism field may result in a child’s crisis, hence, the creation of a robust, reliable and explainable solution is essential. For this reason, the user research took a long and slow journey to assimilate enough knowledge about the context and its peculiar characteristics.
At the end of the user research process, we were capable of defining the AIDA concept: a recommender system that provides caregivers with suggestions on methods and tools to communicate effectively with children on the spectrum.
The next step was the scouting of the most suitable datasets for our concept. Unfortunately, we soon realized that none of the available datasets focused on the communication topic and in particular on the child’s preferences in this respect. Moreover, since we based our concept on caregivers’ knowledge and experiences in real-life situations, the subsequent solution should have been based on the same elements to be truly valuable. Needless to say, we had to create our own dataset.
Key challenges
Creating a meaningful dataset was one of the most challenging steps of our project because the group had to face two main challenges.
Not only quantitative data
First, the dataset needed to be structured from scratch. Hence, AI and design experts have collaborated extensively using a human-centered approach to define key features and data. In doing so, qualitative research played an important role.
Tackling human biases
A second relevant challenge was related to tackling biases. The aim of the user research was not only to identify users’ needs but also to identify potential human biases in advance to build a trustworthy solution.
“When data is gathered, it may contain socially constructed biases, inaccuracies, errors and mistakes. This needs to be addressed prior to training with any given data set. In addition, the integrity of the data must be ensured.”
— Ethics Guidelines for Trustworthy AI by the European Commission’s High Level Expert Group on Artificial Intelligence
Process and tools
How did we work on those challenges? Our approach was guided by three main principles:
- involving domain experts;
- collaborating with AI experts throughout the whole process;
- always getting back to users.
In order to achieve such extensive collaboration, our design process was carried out by leveraging on a broad set of tools and activities — both existing and tailor-made — that are closely related to each other. You can see such interrelations in the picture below.
The user research phase included mostly two activities: interviews and observation.
The role of qualitative data for an AI solution
There are questions that quantitative research can’t answer. Hence, the qualitative research method’s role is to cover this gap by helping data scientists and AI experts in contextualizing the solution (Robyn Rap and Vicky Zhang, Qualitative + Quantitative, Alex Moltzau, The Qualitative Data Scientist).
For this reason, the first phase of the AIDA project consisted of qualitative research aimed at deeply understanding the ASD field and its actors, identifying the needs of children on the spectrum and scouting opportunities for improvements through AI.
We found out that an AI solution can bring unique value by providing caregivers with personalized suggestions reflecting an individual child’s characteristics and communication preferences.
Riding from qualitative to quantitative data
Once collected qualitative data about stakeholders’ habits and pain points, we defined which quantitative data were needed to train the AI model and, consequently, assess the system inputs and outputs.
Starting from several scenarios, the group highlighted the different tools, methods, and information used by different caregivers to communicate with the child. Such elements constitute the machine’s knowledge to caregivers (particularly for parents or new support teachers that have not so much experience) as suggestions in specific situations. In other words, such knowledge constitutes the machine output. As a consequence, it was clear that we needed a dataset composed of tools and methods related to their effectiveness in various situations and for different children. Such information is what we mapped as the desired machine input (see picture below).
The hypothesis of tackling biases through user research
Once mapped the desired inputs and outputs, we started thinking about how to collect such knowledge in a complex environment with different actors characterized by multiple experiences and backgrounds. As a consequence, a parent often has an educational vision and method that may strongly differ from an educator’s or teacher’s. This diversity is what generates the human biases that may be reproduced by the system.
There are many theories about how to tackle biases: some suggest that it is better to maintain a single threshold for every user, while some others suggest that setting different decision thresholds may achieve the best balance.
As a team, we decided to follow the second approach since we hypothesize that it is possible to define different thresholds to weigh human inputs according to their biases by identifying different users’ viewpoints. Hence, we mapped our stakeholders’ biases through a tailor-made tool: the biases table.
The biases table has the purpose of mapping the different viewpoints of multiple actors (or actor categories) on a specific matter. As shown in the picture above, we applied this tool to highlight how each caregiver evaluates different children’s abilities (autonomy, communication, learning, attention, flexibility) according to his background. Differences are highlighted by showing the weight that each caregiver attributes to each topic.
Such weights have been used as a trace for the data collection and model design not to reproduce human biases into the machine. Indeed, data collection will work as a test for the biases table that, by now, should be considered as a first version for the project’s purpose.
Defining key features and data for the machine learning model
At this stage, the group was able to define the key features of the future AI model. To achieve this goal, we created the key features table, another tailor-made tool, to make a common ground among the two expertises. The key features table is a simple table built upon previous activities’ outcomes to clarify two fundamental aspects of the dataset structure: which are the necessary data to train the AI model and from which stakeholder we have to collect each of them.
Ethnographic research for data collection
Once created a common ground for data collection, the team focused on identifying the most suitable instrument to perform this activity. The choice fell on a design probe in the shape of an app diary enabling caregivers and parents to collect and share information about specific events and related child’s preferences.
Why ethnographic research?
Using a longitudinal ethnographic research methodology could help the group collect data about the same user and person over time.
Why a social diary?
The diary can be a useful tool for stakeholders to track and share information about the child, providing a first solution for caregivers to get to know their children better.
Why a digital version?
A digital version of a diary allowed us to create constraints, fundamental to collect structured data for the dataset.
Key Takeaways
Collaborate from day one
In order to design a more effective solution, it is fundamental to collaborate from the very beginning. Collaboration helped both groups in contextualizing the solution, considering both the field’s characteristics and technological constraints.
Quantitative data are not enough
In our experience, starting from users led us to exclude available datasets and create something starting from real actors’ knowledge and environment. Hence, qualitative research allows to achieve a deeper understanding of the context of interest, which is fundamental to create an integrated solution.
Hard challenges lead to the best outcomes
Defining a data structure starting from scratch is difficult but not impossible. In particular, we found out that the ethical approach adopted was helpful in making key choices by testing the data collection tool with final users.
Never underestimate the system’s potential biases
Being conscious of the domain complexity is fundamental to tackle potential biases since the early stages of the project, providing the team with a common understanding of different users’ viewpoints.
Want to discover more on the project? Check out the case study.
Want to start a conversation about AI? Reach me on LinkedIn.
AIDA Team
User Research and Analysis
Anna Focaroli
Lucia Ferretti
User Research
Corinne Schillizzi
Marina Scognamiglio
Michele Armellini
Visual Design
Tamara Ristic
Artificial Intelligence
Saverio D’Amico
Claudia Lunini
Adriano Manfrè
Davide Rezzonico
