Iterative design and development phases (9 months in GIFs)

Weeks 5–8

Below, we summarize the last 9 months on the Lumilo project, letting GIFs do most of the talking. The work so far is divided into broad stages: the “Generative/exploratory stage” of research, the “Formative stage” (which includes various technical groundwork on the project), and the upcoming “Evaluation stage”.


Generative / exploratory stage

  • Analyses of log data and classroom observations from ‘business as usual’ in ITS classrooms — What is? What should be?
  • Interviews with teachers to better understand their challenges in ITS classrooms — What is?
  • Paper-based, generative design exercises and interviews with teachers, to discuss possible solutions to some of these challenges — What could be? What should/shouldn’t be?

Formative design work and technical groundwork

  • Architectural extensions: enabling CTAT/TutorShop to support authoring of custom analytics and learning analytics tools (for teachers and/or students) — now supporting the Lumilo, Luna, and Collaborative Dashboard projects
Current (working) logos for a number of developer tools, APIs, and code repositories, that extend the existing CTAT/TutorShop infrastructure for intelligent tutoring systems into the Learning Analytics space.
  • Creating a public detector library, with initial versions of training materials, for CTAT tutors
  • Narrower affinity diagramming and interpretation sessions — Role prototyping: What should/shouldn’t be?
  • Storyboarding sessions — Role prototyping: What should/shouldn’t be?
  • Iterative, participatory design: Lo-fi experience prototyping [time-freeze] — Role prototyping and information design: What should/shouldn’t be?
Simulating the experience of making decisions based on information presented in ‘teacher glasses’ using PhotoShop layers. Interactively configuring displays based on teachers’ real-time feedback.
Participatory design and early investigations into teachers’ in-the-moment decision-making
….
  • Iterative, participatory design: Mid-fi experience prototyping [time-freeze] — Role prototyping, information design, and experience design: What should/shouldn’t be?
  • Iterative, participatory design: Mid-fi experience prototyping [Wizard of Oz-ing analytics in a simulated lab session, using the TutorShop replay system] — Role prototyping, information design, and experience design: What should/shouldn’t be?
  • Initial analytics development, testing and refining CTAT/TutorShop architectural extensions, and tablet-based interface development (tablet interface is currently used for technical tests, but will be developed into a standalone interface this summer)
Screen capture from a live technical test in a computer lab, running at 3x speed (for each student component, an orange on the left indicates greater that a student has made greater than 3 incorrect attempts on the step this student is *currently working on* ; an orange circle on the right indicates a ‘not acceptable’ value of the Help Model)
An implementation prototyping session: helping a visiting researcher develop the first CTAT/TutorShop real-time dashboard for intelligent tutoring systems that support student collaboration!

Currently in progress:

  • Synthesis of data from prototyping sessions thus far
  • Cognitive task analyses based on teacher think-aloud data
  • Lumilo redesign, informed by both of the above
  • Creating an efficient/sustainable TutorShop-HoloLens link (i.e., push mechanism rather than pull)
  • Developing the first fully-functional head-up display prototype
  • Analytics development and/or refinement, based on redesign

Next steps:

Formative stage (continued)

  • Iterative, participatory design: Mid-hi fi experience prototyping and think-aloud [temporal] —Experience design, alert policies, temporal information design (e.g., persistence of historical/’cold’ information), design of particular analytics

Evaluation stage

  • Lo-mid fi context evaluation studies (more tightly controlled in-lab experiments, investigating teachers’ in-the-moment decision-making)
  • Synthesis of data from evaluation studies and design refinements
  • Classroom evaluation studies
One clap, two clap, three clap, forty?

By clapping more or less, you can signal to us which stories really stand out.