Explanations from alleged masters of shape dissection

We’d just invited a few thousand students to try our latest experiment in online open-ended response activities — a controlled trial with pre- and post-test questions focused on concept transfer. We started reviewing their work, but the data from the control group stopped us in our tracks. We got page after page of responses like this:

This kind of answer wouldn’t be so surprising, except that all of these students had just solved four numerical “find the area of this polygon” problems perfectly, including one with exactly this shape.

We were testing an activity in which students would interact with each others’ open-ended explanations and ideas. They’d learn about finding the area of polygons by dissecting them into simpler shapes.

Our control group watched a video lecture on the subject, then worked through typical textbook-style exercises on the subject, like these:

Plenty of students solved every one of these exercises correctly. Typical standardized tests might suggest that these students had mastered shape dissection. But a third of those students couldn’t give even a partial answer to the “explain why” question above; another third gave poor or incomplete answers.

We didn’t design our experiment to look at this issue: we included these area-finding problems as a “dummy” activity to parallel the richer discussions in our experimental group. Maybe the issue is that the “explain why” prompt involves more algebraic knowledge. Besides, our experimental population is not necessarily representative.

But the intense gap between the procedural and conceptual assessments here still shook me.

Have you ever gotten an “A” in a class but felt you didn’t really understand the material? That seems to be exactly the disparity we’re seeing here. How many numeric-input or multiple-choice tests would exhibit the same problems?

We’ve long discussed our suspicions around scores taken from this kind of problem; we didn’t expect to confront such jarring examples so directly!

Originally published at klr.tumblr.com.




Working notes from explorations into future possibilities in education

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Andy Matuschak

Andy Matuschak

Wonder, blunder, salve, solve! Exploring empowering future possibilities in education with team at Khan Academy, where I lead Early Product Development.

More from Medium

My Struggle with Identity in the Wake of COVID-19

From Covid to Hybrid — How to Keep Moving


Want to smell attractive? Try out these latest perfume trends for the perfect summer vibe