AI Learning Helper
Designing for AI with system design approach
AI Learning Helper is a product designed to help children learn at home. It is developed by NTT DATA Group innovation centres in Denmark, UK and Romania. My role on this project is bringing system design and product design experience to the innovation team.
In this article I will share my approach to designing for AI and using system design methods to develop business propositions. We will look at the methods the team used to develop the product vision, business model ideas and and plan the MVP. The team is developing the MVP and engaging with potential partners now.
Challenge
When the project started in 2019, the goal was to help children, who cannot attend school, learn at home. There are many different reasons why children cannot go school. It may be disability, temporary illness, psychological problems, parent’s or guardian’s choice. Homeschooling is expensive and time-consuming. Often, these children do not receive the same quality of education as children going to school.
In 2020, as the pandemic took hold of most of the world, parents had to become teachers overnight. Suddenly, homeschooling became relevant to everyone. The scope and the potential impact of the project expanded.
It became a society-wide educational challenge.
We united as system design, service design, data and innovation experts to find a solution to this challenge.
How can artificial intelligence help children learn, teachers teach and parents homeschool?
We wanted to develop a scalable solution, a personalised learning helper, that any child, parent or teacher with a connected device can use.
After considering many areas of early education, we chose to focus on learning to read. It is a fundamental skill, which lays foundation for most other types of learning.
Research
NTT DATA has a partnership with MIT Media Labs. It gives us access to a treasure trove of research insights into innovative applications of advanced technologies. We reviewed research in the field of using robots for early education and social companionship and AI augmented learning. The key concepts of the product are based on the following learnings.
Social learning
Children in early primary school learn socially. The relationships are very important for their engagement and motivation.
Children have better learning outcomes and knowledge retention when they have an emotional connection with their learning companion.
Research has shown that children respond very well to learning with a buddy character. Many existing solutions focus on testing children’s knowledge and finding the right content. Social and emotional components are often overlooked.
Pedagogical methods for learning to read
We conducted desk research and interviewed teachers to understand key pedagogical methods. It led us to ”The simple view of reading”, a widely accepted scientific theory, explaining how reading skills are developed. A student’s reading comprehension depends on pronunciation (decoding) and understanding (language comprehension):
Decoding (D) x Language Comprehension (LC)= Reading Comprehension (RC)
We chose to use this formula for evaluating learning progress and recommending next steps.
We reviewed typical reading lesson plans, discussion guides and activities teachers use to introduce a book. These materials provided guidance and inspiration for a range of teaching methods used by the AI Learning Helper.
Homeschooling challenges: interviewing teachers, parents and kids
To understand the challenges of remote learning, I interviewed children, teachers and parents. Below are the highlights of what we learnt.
The teachers find it hard to keep students engaged remotely. It is difficult to track learning progress and give individual attention and support to each student.
The children miss learning with friends, feel less motivated and more distracted at home. They liked the idea of learning with a buddy, like an older brother or sister and commented that “it would feel less lonely”. Having their favourite game, book or movie characters as their learning buddy avatar would make learning more fun.
Parents commented, that they struggle with playing a teacher role, and balancing it with other commitments. They would want to receive guidance on the best way to teach their child and have more visibility of how well they are doing.
Approach
We set out to create a personalised AI Learning Helper, that can develop an emotional connection with every child. It should evaluate their learning progress and adapt teaching strategy. We wanted to make learning to read fun, using edutainment and gamification. The solution needs to be scalable and accessible to anyone with a laptop or a tablet.
We designed the AI Learning Helper to have 3 layers:
- Teaching engine — a platform that can import educational content and help children learn to read and understand it
- Emotion AI — a learning companion that detects emotions and responds in a human-like way
- Engaging AI Learning Helper avatars that can be part of an app, exist in a 3D learning environment or in a physical space
Teaching engine
AI provides a way to codify pedagogical methods and best practices for learning to read. We developed AI capabilities to support key skills important for learning to read. For each book AI Learning Helper provides real time pronunciation feedback. It can test how well children can summarise text, describe images, ask and answer questions about the text.
The AI Learning Helper is evaluating student’s learning progress to recommend the best next steps. We are planning to continue refining the evaluation criteria in collaboration with teachers.
To make the solution scaleable, we aim to minimise human effort when importing a new book into the engine. Each book will be “onboarded” once and made available to all users. AI capabilities supporting pronunciation and understanding will be added to enrich content. Eventually, the book onboarding process will be automated. For the MVP some of the AI capabilities will be done or reviewed by humans.
Emotion AI framework
Our NTT DATA Group colleagues at the itelligence Transformation Lab in Denmark have developed it.human platform over the last couple of years. They offer digital human employees companies can hire for providing customer service.
The it.human platform became an integral part of the AI Learning Helper. It enables the AI Learning Helper to have a natural conversation with users and build an emotional connection. The AI Learning Helper monitors facial expressions, vocal intonations and body language. It detects emotional reactions and responds in a human-like way.
Engaging helper avatars
To improve engagement and motivation, the AI Learning Helper can look like children’s favourite characters. We designed the solution that is flexible enough to work with many different types of avatars. The it.human platform is able to work with many types of 3D models. It can bring the avatars to life, giving the ability to speak, move, understand and express emotions.
Designing for AI
Bringing together AI, design and business
Our multi-disciplinary innovation team worked in close collaboration throughout the project. We combined our research insights from AI, business and human perspectives to co-create the product vision.
Behind our friendly learning buddy is a carefully orchestrated system of AI components. AI experts in 4 countries from Northern Europe to Australia are developing it. Defining an end-to-end user experience was essential to align the team around the product vision and coordinate work. I created Storyboards for the product vision, the MVP and the vision video.
Testing AI Learning Helper
We are testing the product in stages:
- Making sure it works as expected, and is intuitive and easy to use
- Get feedback on the learning activities, choice of avatars, voices etc.
- Responding to children's behaviour and emotional reactions
- Evaluating learning progress, adapting teaching methods and choosing the right content
- Improving learning outcomes
Often in early tests many aspects of the experience are imitated by a person. Early feedback is valuable for AI teams to focus development efforts and consider real life scenarios that might be less obvious. Gradually they are replaced with AI algorithms.
Testing prototypes with children gave us many surprising insights. The way they interacted with the product and the questions they asked often were unexpected. It helped us understand what is important from a child’s perspective and provide input for UX and conversational design.
The focus of this article is on the design process. To read about user research for AI and collaboration between AI and design teams at NTT DATA Group in more detail see these articles:
AIDA: the challenge of user research for AI in the autism field
Using qualitative research to structure an unbiased AI dataset
Thoughts on the role of Designer of AI products and services
The role of an AI Designer is a Translator between human needs and advanced technology. Let’s look at some aspects of this role:
- Understand the art of possible, the potential and limitations of AI and robotics now and in the near future
- Invent genuinely useful applications of AI, based on solving real needs, humanising technology
- Distill knowledge and codify experience of experts, collaborate with subject matter experts throughout the project
- Test with humans, who are going to use the service, carefully planning what can be tested at each stage
- Design responsible AI, considering ethics of AI, handling sensitive data, as well as longer term consequences
- Define enablers and orchestrate backstage actions for each aspect of the experience
Designing how to get the product info in the hands of users is as important as creating a great product.
System design approach
Helping children learn to read is a great example of a problem that cannot be solved by a single company. Many people and organisations support children on their journey to becoming confident readers.
System design offers methods and tools to create solutions, delivered through partnerships. I will share the system design approach we used for AI Learning Helper.
The strength of our team is in design and ability to develop cutting edge AI solutions. We do not specialise in education or entertainment. Partnerships in these areas are important to create this product.
To teach children reading skills, we need educational books. We can work with publishers and authors to offer them an innovative distribution channel and provide books to users.
To deliver the best learning outcomes for children, AI Learning Helper needs to codify the best pedagogical methods. Input from leading educators and academia would benefit the product.
To make the learning helper avatars more engaging, we can make them look and behave like children’s favourite toys, game, book or movie characters. Collaborations with companies, which own characters children love would benefit the product.
Teachers and parents are the best people to introduce the AI Helper to children. Organisations working with schools and parents can deliver the product to their customers.
For the AI Learning Helper to succeed, we need to build an ecosystem. A carefully orchestrated partnership of organisations contributing to and delivering the service.
We started with understanding the product ecosystem.
Product Ecosystem
Product ecosystem is a network of organisations, co-creating, delivering or consuming a product. It may include suppliers, partners, distributors, customers, competitors or government agencies.
In service design, we map an existing Ecosystem, to understand the context a service operates in. In system design we design a new Ecosystem that can deliver a service through partnerships of organisations.
Benefits and outcomes
- Identify potential partners and clients, understand all parties involved in creating, delivering and consuming a product or service
- Consider all types of value stakeholders can exchange, if they participate in the product ecosystem, what each of them can contribute and receive in return
- Provide context for creating business models, partnership agreements and positioning the product to potential clients and partners
I created an Ecosystem Map to visualise our collective knowledge of the stakeholders and the values they exchange. The Ecosystem Map helps to understand the big picture, the lay of the land, all potential partnerships the product team can have.
Value flows —Business models
I created the value flow diagrams to visualise potential business models.
Value flow (or value stream) shows how value is added to a product or service over time all the way to delivering it to customers.
Benefits and outcomes
- Define stages, where the value elements are added to the product over time
- Show how value elements can be added at each stage by potential partners
- Consider partnership models and financial agreements between different parties
We created many different scenarios. Each value flow shows a potential business model, collaboration with a different set of partners.
This approach helps to quickly model, tweak and evaluate business ideas, illustrated in a simple visual way. It is also a good way to communicate proposals to potential clients and partners.
First, we map out the stages of the value flow and what types of value are added to the product at each stage. Then, we show what our team can contribute. The remaining areas of value are the opportunities for collaboration. The next step is to consider possible partnerships, using the Ecosystem Map as a reference. Below you can see a few examples.
Planning MVP
To define the MVP experience we used a Storyboard and a Blueprint, optimised for a conversational AI product. This type of storyboard has some additional elements.
Storyboard for conversational AI
Benefits and outcomes
The usual:
- Create a user journey incorporating features essential for MVP
- Define UX and UI requirements
And also:
- Create a conversational design flow for each feature. Include the kind of questions and answers the AI Learning Helper should be able to respond to.
- Plan responses to different types of user behaviour. For example, how the AI Learning Helper should respond when a child is distracted or bored or talks about something random and unrelated. When a child is excited and wants to talk about something related to the book from their personal experience.
- Define enablers of each feature. Include AI components, algorithms, platforms and data requirements, backstage activities completed by humans.
Blueprint for conversational AI
Blueprint shows everything that needs to happen to create and deliver a product or service. A Blueprint shows actions of people interacting with customers or working behind the scenes. It also shows resources required, including materials, technology and data.
For the AI Learning Helper MVP, I created 2 Blueprints for the areas most pertinent to the team:
- MVP development Blueprint. For each feature and AI capability, defined activities, needed to develop them. They included AI, data, business development, UX, research, front end and back end
- Book onboarding Blueprint, showing what needs to be done to add a new book to the AI Learning Helper
Benefits and outcomes
- Plan product development activities for each feature and AI capability
- Define data journey, data types needed for each steps of the journey, how it needs to be captured, stored, processed and used to deliver the service. How data can be used to continuously improve the product.
- Partner engagement, show partner engagement at every stage
The areas that are useful to add for this type of Blueprint are Data journey and Partners.
Next steps
The team is building the MVP and engaging with potential partners now.
We will be testing the MVP with children, teachers and parents in England and Denmark, to continue refining the product.
Forging the right partnerships is an intrinsic part of developing a system design solution. Our next steps include crafting partnership models and commercial models for each type of partner. Once we find potential partners, we need to develop a proof of value for each of them, before preparing for implementation.
Conclusion
In this article we looked at the part of the project where we co-created the product vision, leading up to the MVP.
We used a system design approach to develop the AI Learning Helper as an open and flexible solution. It will support partnerships with different types of organisations in the learning ecosystem.
Commercialising the product will bring a new set of challenges. We can take advantage of system design methods to address them. For example, developing proof of value for each partner and refining the value flow. Crafting partnership models and financial models, and partner pilot frameworks. This can be a topic for another article.
System design offers methods for bringing together advanced technology, business and product design. It empowers us to take on complex challenges and design solutions delivered though an ecosystem. It helps to orchestrate partnerships of organisations aligned around a shared goal.
