CMU LearnLab Summer School 2017: Innovation, Understanding, Iteration

Benji Xie
Benji Xie
Jul 24, 2017 · 4 min read
Catching CMU on a sunny day before many days of thunderstorms.

I’m driven by the impact of research discoveries. But there’s this frustrating gap between research discovery and practical implementation. I suspect this gap is because researchers and practitioners lack the infrastructure and opportunities to collaborate. And addressing this gap is critical for my current research, where I am developing a formative assessment tool to improve introductory computer science education. So, I went to Carnegie Mellon University to connect with hardworking people thinking about similar challenges and see how the LearnLab was bridging the gap between educational research and practice.

The LearnLab at CMU exists at the critical intersection between educational research and practice. It is a cross-disciplinary laboratory for advancing the practical science of how students learn. To me, it’s a place where experimentation and iteration are interwoven with instructional design. How students think and how technology and pedagogy impact them is at the core of LearnLab’s focus.

I decided to attend LearnLab’s week-long summer school to feed a recent interest and research endeavor in developing an intelligent tutoring system that supports students enrolled in introductory computer science courses. At this summer school, I interacted with professors, students, instructors, and support staff working to advance both their own courses as well as our understanding of how students learn. Half of our time was spent hearing from LearnLab faculty and students about topics including transfer and learning, cognitive task analysis, and educational data mining. The other half was spent in our specific tracks (I was in the intelligent tutoring systems track), working towards a poster and presentation at the end of the week.

This workshop reminded me why I love what I do: The community of educational researchers and instructors is full of genuine people who are driven by their passion to empower others. I discussed with Ken Koedinger (CMU) how to balance the burden and benefit of self-regulation tasks in pedagogy. I had a really engaging conversation with Kalyan Veeramachaneni (MIT) about data science education and how there is an overemphasis on feature engineering and model selection and an underemphasis on the “human” elements: developing prediction questions, identifying relevant data, interpreting results, and making decisions. Vincent Alevan (CMU) shared his perspectives on designing effective educational experiences by focusing first and foremost on what’s hard for students to think about, a mantra I will use to inform my practice moving forward.

Key Takeaways

  • You can’t see learning happening! Students don’t know what they know. Much of this knowledge is tacit. Furthermore, instructors suffer from expert blind spot, where they have too much expertise to know about learner difficulties and how to address them. Cognitive task analysis helps decompose what we want to teach into components and understand “hidden skills” to better teach these concepts.
  • Educational technology is about using data to drive iterative improvement. If we collect and analyze data that represents the desired learning outcomes, we can identify shortcomings in a course and improve upon them. Relying on hunches is not rigorous. Simply applying learning theories with blind trust is also ill-advised because often times the domain is different and the theory may or may not apply.
  • General transfer learning is a fantasy. The notion that the mind is a muscle that is exercised is overly simplistic and wrong. Knowledge is largely domain specific, so transferring knowledge between situations is hard. We can achieve some intermediate transfer, perhaps through teaching general strategies or explicitly supporting self-reflection.
  • Adaptivity can happen at different time scales and for different types of knowledge. We often have a very narrow perspective of what it means to be “adaptive” (no doubt influenced by Siri, Alexa, Cortana, Google Home, etc. adapting to our usage). Adaptivity can happen at different time scales (e.g. providing relevant feedback after an incorrect action, updating curriculum for the next course offering). It can also apply to different types of knowledge! (See the table below.)
Adaptivity Grid: Adaptivity can happen at different time scales (between classes, between tasks/lessons, between steps in a lesson), for different types of knowledge. Adapted from Table 1, Aleven, McLaughlin, Glenn, & Koedinger 2017

My Ongoing Research: Supporting code tracing by embodying the computer

The program tracing tutor I worked on. Here, students have to click on the next line of code that executes while also updating a table that keeps track of the variables within each method. The tutor provides step-by-step feedback in the gray box.

Final Thoughts: Don’t just apply learning science. DO learning science!

We can create more accessible and effective learning experiences if we use information to test assumptions and improve iteratively.

Bits and Behavior

This is the blog for the Code & Cognition lab, directed by professor Amy J. Ko, Ph.D. at the University of Washington. Here we reflect on what software is, what effects it's having on the world, and our role as public intellectuals in help civilization make sense of code.

Thanks to Harrison Kwik, Leanne Hwa, Alex Tan, and Amy J. Ko

Benji Xie

Written by

Benji Xie

Ph.D. candidate at the UW iSchool. There’s a symbiosis between man, machine, and data; I’m all about it. @benjixie

Bits and Behavior

This is the blog for the Code & Cognition lab, directed by professor Amy J. Ko, Ph.D. at the University of Washington. Here we reflect on what software is, what effects it's having on the world, and our role as public intellectuals in help civilization make sense of code.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade