The Lumilo Project

Week 0

Many automated, real-time detectors of student learning and behavior have been developed in the Educational Data Mining and Learning Analytics communities over the last decade. However, these advanced analytics are typically used to drive automated interventions by intelligent tutoring systems, rather than to support and empower human teachers.

The Lumilo project aims to reframe intelligent tutoring systems as classroom sensors, which can help augment teachers’ perceptions of student learning and behavior in their classrooms in real-time. This can be viewed as one of several recent efforts towards ‘evolution’ in research on intelligent tutoring systems, moving towards a greater consideration of the contexts (e.g., classrooms) in which these systems are often used [Roll & Wylie, 2016]. Indeed, a shift in the framing of intelligent tutoring systems’ primary role in the classroom — from that of ‘artificial intelligence’ to that of ‘augmented intelligence’ or ‘intelligence amplification’ — can increasingly be seen not only in the research literature [e.g., Baker, 2016; Holstein, Aleven, McLaren, 2017a] but also in popular educational technology media [e.g., https://www.edsurge.com/news/2016-10-06-why-education-needs-augmented-not-artificial-intelligence].

By augmenting teachers’ perceptions of student learning and behavior within educational software, we ultimately hope to support and improve their on-the-spot, pedagogical decision-making in blended class sessions. The notion that classroom use of intelligent tutoring systems can free up teachers’ to serve as “guides by the side” rather than “sages on the stage” is not new in itself. The Lumilo project explores whether and how intelligent tutoring systems could be designed to better help (and be helped by) human teachers. As part of this work, we are currently working on the design and evaluation of real-time, wearable cognitive augmentation for K-12 teachers.

In this early concept image, a teacher puts on mixed reality glasses, and can immediately see additional information about individual students, as well as the class as a whole. At the far left, a “Zzz…” is displayed over a student’s head to indicate that this student may currently be off-task. Some students have raised-hand symbols next to them — effectively representing cases where the intelligent tutoring system is “asking the teacher for help” (given that it has detected it is unlikely to be able to help that student learn certain material, without the teacher’s assistance). At the top right corner of the teacher’s field-of-view, she can see information about common errors students are making in the class, overall.

Work on Lumilo began in the summer of 2016, as one branch of a larger project that aims to design support tools for K-12 teachers who use intelligent tutoring systems in their classrooms. From 2015 - 2016, this broader project primarily focused on the design of an information dashboard, called Luna, to support such teachers in using analytics generated by intelligent tutoring systems to inform their lesson planning, in between blended class sessions (a use case we call the “next-day scenario”). Notes on our process and some early findings from this project be found in the following papers: [Aleven, Xhakaj, Holstein, & McLaren, 2016; Holstein, McLaren, & Aleven 2017a; 2017b; Holstein, Xhakaj, Aleven, & McLaren, 2016; Xhakaj, Aleven, & McLaren, 2016].

Findings from early design and evaluation studies with Luna have informed our current work on Lumilo, but have also revealed important differences in the kinds of support teachers need in real-time, to support on-the-spot decision-making during a blended lab session, versus after-class for reflection and lesson planning.


In our early design work with K-12 teachers (Holstein, McLaren, & Aleven, 2017a), we first aimed to explore their needs for real-time support in blended classrooms more broadly. In particular, we were interested in better understanding the particular teacher needs that existing intelligent tutoring systems and learning analytics systems serve, as well as those they fail to meet (or in some cases, perhaps, even create).

Excerpt from a digital affinity diagram, synthesizing teachers expressed needs for real-time support from intelligent tutoring systems, as well as difficulties they’ve faced with prior learning analytics systems (including reporting systems for intelligent tutoring systems).

We also elicited key teacher needs for real-time support in blended classrooms, by asking teachers:

If you could have any superpowers you wanted, to help you do your job… what would they be?

To help us better understand key opportunities for real-time analytics intelligent tutoring systems to better support teachers, we asked teachers to sort these superpowers by relative priority in a card-sorting exercise, and to discuss their reasons for listing particular superpowers and prioritizing them in particular ways.

Based on the superpower hierarchies teachers generated, as well as findings from our prior interviews with teachers, we then speed dated a range of futuristic concepts through storyboarding sessions — uncovering a number of potential boundaries (and also discovering that some of the boundaries we’d anticipated — especially regarding teachers’ concern over student privacy– may not actually exist).


Gena joined the Lumilo project in February 2017, at a transition point between the largely generative design studies described above, to more focused explorations of particular design spaces. With so many possible directions and design opportunities suggested by our prior work, we were a bit paralyzed at first — faced with the task of exploring a very large space, in limited time! Ultimately, we decided to focus our work this year on the exploration of a relatively uncharted design space, borne out of our past year of participatory design work with teachers: smart glasses that can augment teachers’ perceptions of student learning and behavior in real-time.

Upcoming blog posts will cover our first explorations into the design of smart glasses for K-12 teachers, which could ultimately facilitate more effective pedagogical decision-making in real-time.

References

Aleven, V., Xhakaj, F., Holstein, K., & McLaren, B. M. (2016). Developing a Teacher Dashboard For Use with Intelligent Tutoring Systems. In 4th International Workshop on Teaching Analytics at EC-TEL 16.

Baker, R. S. (2016). Stupid tutoring systems, intelligent humans. International Journal of Artificial Intelligence in Education, 26(2), 600–614.

Holstein, K., McLaren, B. M., & Aleven, V. (2017a). Intelligent tutors as teachers’ aides: exploring teacher needs for real-time analytics in blended classrooms. In Proceedings of the Seventh International Learning Analytics & Knowledge Conference (pp. 257–266). ACM.

Holstein, K., McLaren, B. M., & Aleven, V. (2017b). SPACLE: investigating learning across virtual and physical spaces using spatial replays. In Proceedings of the Seventh International Learning Analytics & Knowledge Conference (pp. 358–367). ACM.

Holstein, K., Xhakaj, F., Aleven, V., & McLaren, B. (2016). Luna: A Dashboard for Teachers Using Intelligent Tutoring Systems. In 4th International Workshop on Teaching Analytics at EC-TEL 16.

Roll, I., & Wylie, R. (2016). Evolution and Revolution in Artificial Intelligence in Education. International Journal of Artificial Intelligence in Education, 26(2), 582–599.

Xhakaj, F., Aleven, V., & McLaren, B. M. (2016). How teachers use data to help students learn: Contextual inquiry for the design of a dashboard. In European Conference on Technology Enhanced Learning (pp. 340–354). Springer International Publishing.