Michael Horn on Innovating in Assessment for Learning

Last week, I sat down with Michael Horn, speaker, author, and thought leader on the future of education to talk about education equity, the rise of A.I., and closing the loop on assessment for learning.

IDVW: In your prize-winning 2008 book “Disrupting Class” you predicted that by now every K-12 student would be receiving a customized, personalized learning experience driven by technology. 11 years later, that hasn’t happened. Why not?

MH: In our book, we assumed that as digital learning grew, so too would customization, or personalization. Although I think our predictions around digital learning were reasonably accurate, although personalization has happened to a degree — and I’d say far more than was happening a decade ago! — we were naive on a number of fronts. First, digital learning has grown within today’s time-bound, factory-model education system, so it has had difficulty transcending and redefining it. We have 6 barriers that I continue to worry about:

1) Assessment — We still rely on summative assessments instead of near real-time, interactive assessments that are both of learning and for learning

2) Policy — We are still in a time constant, student learning variable system as opposed to a competency-based one that focuses too much on mandating the inputs and processes of how to do schooling rather than focusing on the outcomes we want to see and freeing up educators on the ground to figure out how to get there.

3) Teacher and leader preparation and support — We are still not preparing and supporting our educators adequately for a very different paradigm and not giving them the design-thinking and innovation tools to create new models of learning that leverage digital learning, rather than thinking all we have to do is train them on how to use digital tools in the current system, which will never create the change we need.

4) School-building design — We’re still building mausoleums to the past as opposed to leaner, greener, more purposeful buildings that look nothing like traditional schools.

5) Rigor — As we move to personalization, all too often we forget that student learning and work should still be rigorous.

6) Parent and community buy-in — All too often educators who want to innovate forget to bring along the community and help them understand why the innovation will solve their individual problems. Parents are often the most conservative force in schooling, especially if school worked for them.

IDVW: Education is an area of rapid technological innovation. Where do you think technology is having the most positive impact for students? Where is it not paying off for students?

MH: I think there have been amazing strides in areas like math learning, connecting students to high-quality resources around the world, and some personalization in pockets of schools around the country. The introduction of electronic whiteboards and tools of that nature have not been terribly helpful when they have simply doubled down on existing and traditional models of whole-class learning. I’d also add that while credit recovery continues to be an amazing opportunity for innovation, because we have not had robust assessments to verify whether students have learned what we want them to, it seems instead as though the advance of online learning here has been a waste that isolates too many learners.

IDVW: What about equity? Technology costs money and depends on teachers and communities who are familiar with how to use it. Is technology a threat to education equity? Are there areas where technology is clearly driving equity and helping to close achievement gaps? What technologies are helping to promote equity in education?

MH: I moderated a panel recently where there were some serious concerns expressed around this, but overall I actually think technology has been an unbelievable leveller of the playing field. Too often I see well-off districts say what well-run companies tend to say, namely, “Why should we innovate? We’re doing great!” And so they stand pat. Meanwhile, urban and rural districts often have to innovate because they need to do something different to better serve students. Relative to other interventions out there, technology is often incredibly inexpensive, and the prices are constantly falling. The best innovations I’ve seen tend to be those serving the students who I would argue need it the most.

IDVW: What about crowd-sourcing, human computation, and machine learning in assessment technology? Are there innovations in assessment that can improve access and equity in education?

MH: I think to this point we’ve seen some attempts at leveraging these tools, but they haven’t had the impact they still might. Education is an enormously complex endeavor and it’s hard to reduce it in my mind to simple algorithms. That said, I see enormous promise in your work at Validated Learning — leveraging these tools to validate assessment items far more frequently and cheaply that teachers have created such that they aren’t doing, say, A.I. on a student’s response, but on the item assessing the response that a teacher has created and that incorporates a teacher’s insight and connection to the curriculum. These tools need to swim with educators in the field if they are to be useful, not swim against their judgment or try to compete with the best experts.

IDVW: Thank you. I’m also a believer in leveraging A.I. to support teachers’ expertise and unique abilities to teach and engage students. Let’s talk about innovation and disruption in the assessment side of K-12 education. There has been a lot of consolidation of technology companies in the K-12 space. School districts are looking to simplify their technology stack. Does this hurt innovation in assessment for learning?

MH: Good question, and obviously difficult to go into everything given the complexity and variation in the market. What I would say is that I think there are many promising disruptive innovations that have upended the oligopoly the textbook publishers have had for a long time in this market, where before those entities swallowed up promising companies and then the innovations all-too-often withered on the vine. Getting a common assessment engine in place that allows for a wide variety of assessment systems in schools would also help with the amount of tools out there that can overwhelm anyone by essentially telling an educator, “hey, pick the tool best for you, but also use this objective, third-party platform to give you a true-north view if the tool you picked is really helping the students you hoped it did.”

IDVW: Can you talk more about the idea of a common assessment engine? Schools and teachers use a wide variety of learning products, and many of them are from smaller companies. Some of these learning products offer their own assessment tools, and some rely on teachers to transcribe assessments into whatever tool they prefer. So even though a core system like an LMS may offer the ability to acquire assessment data from various assessment tools, we still don’t have a common way to get the assessment questions from the learning tools into the assessment tools. Can you talk about how this problem might be solved?

MH: Right, this is an incredibly complicated problem. First, we don’t know if the assessments are from different tools are measuring the same thing they purport to, and we don’t have a way to get the questions from tools that have made their focus around content — not assessment — into assessment tools. It seems to me this is where you all can make a big impact through the learning exchange you’ve pioneered that can help put these on a common platform AND validate them to make sure that one set of assessment items from one provider that purports to align to certain standards is indeed measuring the same thing as a different set of questions from a different provider. This isn’t alignment, it’s validation. So we need both a technical solution and a learning solution to this problem.

IDVW: What else needs to change in order for assessment to become a driver of teaching and learning at scale, rather than a tool for measuring outcomes?

MH: Great question. Again, there’s a reason why I listed assessment as my top concern in the first question. I just don’t think rigorous personalization happens without robust, valid assessment that is smaller, more frequent, and both of and for learning in the future. And we’ve had very little system-wide innovation — or even frankly attempts at cracking this nut. If you don’t know how each child is doing at any point in time, fundamentally how can you meaningfully personalize for what each child needs? You can’t. So we need to step it up as a nation and invest in assessment.

IDVW: Thank you very much, Michael! This has been illuminating.

MH: You are welcome!