Curiosity Notebook: The Design of a Research Platform for Learning by Teaching

This post summarizes the paper “Curiosity Notebook: The Design of a Research Platform for Learning by Teaching”, by Ken Jen Lee, Apoorva Chauhan, Joslin Goh, Elizabeth Nilsen and Edith Law, on research done in the University of Waterloo, Canada. This paper will be/was presented in ACM CSCW 2021. Paper available at https://doi.org/10.1145/3479538 or https://arxiv.org/abs/2108.09809.

Curiosity Notebook’s interface.

Ever wondered what it would be like to learn something by teaching an agent, be it a chatbot or a physical robot? In this work, we present the Curiosity Notebook, a new research and educational platform for learning by teaching in various learning settings.

Learning by teaching is, as the name suggests, when students learn by teaching others. But what is so unique about it? Learning-by-teaching’s protégé effect — students learn more effectively by teaching others. Benefits include better long-term understanding, a stronger ability to identify gaps in their own understanding, increased motivation, and many more.

Even though learning by teaching could be significantly beneficial for education, it is very hard to research learning by teaching with only human students. For example, if you have a classroom of students, how can you pair them into tutor (the student who is teaching) and tutee (the student who is being taught) pairs such that all pairs are consistent? Each tutor-tutee pairing might have varying differences in understanding (how much the tutor knows vs. how much the tutee knows), relationships (friends vs. strangers), teaching styles (e.g., patient vs. impatient), learning styles (e.g., examples first vs. definitions first) etc. In other words, it is hard to conduct learning-by-teaching studies with human students due to the many confounding variables. Moreover, real human students might be harmed in studies because they are paired with tutors who (intentionally or not) taught them incorrect information, treated them disrespectfully, demotivated them, etc.

So, what can we do about this? One solution is to use tutees who are not humans, but software programs, or agents, as we call them. Since these agents are built to be taught by students acting as tutors, they are referred to as “teachable agents”. This is like having every tutor teach a copy of the same exact tutee, which solves many of the problems listed above.

However, we are not done yet. In the real world, learning takes place in many settings, for example, online vs. in-person, university vs. elementary schools, one-to-one vs. multiple people teaching a single person. As such, if we were going to build a teachable agent platform that can be used to research learning by teaching in these various settings, what features would such a platform need to have? Curiosity Notebook supports these 5 configurable features (CF), which were designed iteratively over 2 deployments:

• CF#1: Agent Characteristics. The platform supports agents that can be configured to have different characteristics. For example, Ceha et al. [1] configured agents on the Curiosity Notebook to have different humor styles and found that affiliative humor could significantly increase student tutors’ motivation and effort, while self-depreciating humor could negatively impact their enjoyment.

A conversation with a humorous agent in a study using the Curiosity Notebook by Ceha et al. [1].

• CF#2: Quantification of Teaching Strategies. Curiosity Notebook supports 7 unique teaching conversations, which can be initiated by student tutors using the 7 buttons in the interface. Tutors can flexibly choose how they teach the agent, and researchers can accurately observe and measure the tutors’ behaviors to better explore their research questions.

Unique functionality of each of the 7 buttons in the Curiosity Notebook.

An example Describe conversation where the agent learns about Schist.

• CF#3: Scalable Learning Task and Material. Curiosity Notebook supports easy configurations of the learning task and material (e.g., elementary school material, university-level material) via the admin interfaces.

Curiosity Notebook’s admin interface for configuring the material shown in the interface.

• CF#4: Coordinated Group-Based Teaching. The platform supports having one or more tutors teach a single agent, and provide configurations for how the group interacts (e.g., does the agent explicitly call each tutor to teach it?). This allowed us to quickly deploy to Deployment 1 (left image below), where elementary school students taught agents in groups of 2 or 3, and Deployment 2, where university students taught agents individually. Moreover, Ravari et al. used Curiosity Notebook for a study (right image below) where the agent used reinforcement learning to encourage more dialogue between pairs of participants who taught the agent together [2].

Deployment 1, carried out in an elementary school where groups of 2 or 3 students taught a NAO robot (left), and a study by Ravari et al. [2], where pairs of university students taught NAO together (right).

• CF#5: Flexible Agent Embodiments. The platform’s agent is able to take many forms, be it a physical robot (e.g., the NAO robot like in both images above), or a text-only chatbot (like in Deployment 2).

Moreover, to show the student tutors what the agent has learned, the agent keeps a notebook, which is updated live during teaching sessions.

Table of content page of the agent’s notebook (left) and the page containing notes for a rock called Slate (right). The notes reflect everything that the agent has learned from the student tutor(s).

What next? We identified a few possible improvements for the Curiosity Notebook, and have plans to release the code publicly so that it serves the purpose it was built for — to make it easier for researchers to study learning by teaching in various settings.

If you are interested in using Curiosity Notebook, exploring or extending its capabilities, please read the full paper (links below) and we look forward to being contacted! We can be reached at edith[dot]law[at]uwaterloo[dot]ca (Dr. Edith Law, Principal Investigator), or kenjen[dot]lee[at]uwaterloo[dot]ca (Ken Jen Lee, First Author).

Paper links: https://doi.org/10.1145/3479538, https://arxiv.org/abs/2108.09809

Full citation: Ken Jen Lee, Apoorva Chauhan, Joslin Goh, Elizabeth Nilsen, and Edith Law. 2021. Curiosity Notebook: The Design of a Research Platform for Learning by Teaching. Proc. ACM Hum.-Comput. Interact.5, CSCW2, Article 394 (October 2021), 26 pages. https://doi.org/10.1145/3479538

References:

[1] Jessy Ceha, Ken Jen Lee, Elizabeth Nilsen, Joslin Goh, and Edith Law. 2021. Can a Humorous Conversational Agent Enhance Learning Experience and Outcomes? Association for Computing Machinery, New York, NY, USA, 1–14. https://doi.org/10.1145/3411764.3445068

[2] Parastoo Baghaei Ravari, Ken Jen Lee, Edith Law, and Dana Kulic. 2021. Effects of an Adaptive Robot Encouraging Teamwork on Students’ Learning. In 2021 30th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN). ACM, New York, NY, USA, 8. https://doi.org/10.1109/RO-MAN50785.2021.9515354