The World Bank’s new tool to measure classroom success
Some context
Teach was recently launched by the World Bank Group with a view to measuring effectiveness inside classrooms in the developing world.
“Teach is an open source classroom observation tool that provides a window into one of the less explored and more important aspects of a student’s education: what goes on in the classroom.”
It’s a thorough, ambitious project with an admirable focus on the minute-by-minute experience children have in schools. So far, it has launched a lot of productive discussion, much of it high-level thinking through of the policy implications of the tool. Excellent dialogue; lots of interesting points of view.
My concerns here are a bit narrower — namely, how it might influence better, more coherent instructional design, and how that influence might shape a kid’s day. In short — with the rollout of Teach, what changes about Sarah’s experience of sitting in a 40-minute lesson? How does the tool help her learn more than she would have otherwise?
To help answer, let’s look at the Teach framework -
- and zoom all the way in on “Instruction,” and more specifically “Lesson Facilitation,” something I was very curious about how they defined and measured.
Here’s what it looked like in the rubric:
AND
Great. The teacher’s explanation must be clear (NB: here’s the best writing I’ve ever seen on “clear instruction” — Shepard Barbash on Zig Engelmann) and the teacher must model the process of thinking. These strike me as exactly the right things to focus on within “Lesson Facilitation,” and are connected to a problem we’ve often wrestled with: What’s the best way for a teacher to explain or model something so that it sets up a productive “Independent Practice”…?
Before exploring that, let’s stipulate at the outset that (i) the vast majority of primary school children in low and middle-income countries don’t get enough productive practice over the course a typical 40-minute lesson, (ii) this is explained in part by inefficient pedagogy, or in this case, sub-optimal “Lesson Facilitation.” We can also agree that lesson facilitation looks different in different pedagogical contexts. Teaching kids how to decode requires a different facilitation than conducting an experiment about, say, sound and light.
Let’s walk through an example with that in mind. Imagine you’re trying to develop an understanding of how to solve two-step algebraic equations, and you’ve got a 40-minute introductory lesson to do it. You start with the problem “2x + 5 = 15,” and map back from a goal of solving for x, or isolating x, or, as students often say, a bit imprecisely but endearingly, “getting x by itself.”
There are lots of details to obsess over in figuring out the best way to get kids to learn this very procedure. Five quick points of context:
1. “2x + 5 = 15” is a “two-step equation,” as opposed to “one-step.” This means it takes more than one operation to solve for the variable. You’re assuming that kids have some background in solving for an unknown. They’ve seen and successfully solved one-step problems like:
x + 4 = 10;
12 = x + 3.
They hopefully have some sense what an “unknown” is, conceptually.
2. Kids are starting to scratch the surface of “equivalence” via some good, low-level algebraic manipulation. At minimum, they can utter something like “whatever you do to one side, you have to do to the other.” We’ll take that as a novice-level understanding that will evolve over time.
3. Remember the typical teacher in low and middle-income countries doesn’t have much content knowledge. (Not their fault, David Evans argues.) Nor do they have much pedagogical content knowledge. (A complicated problem, Tessa Bold and team argue.)
4. Against that backdrop: how do we get kids safely into independent practice, safely into the zone of “productive struggle” …. where they’re struggling just enough but not too much? Think: a novice weightlifter moving from 105 lbs to 110 lbs on the squat type-of-struggle. Not from 105 to 350, where they fall over, injure themselves…without getting stronger.
Which brings us back to…
5. 2x + 5 = 15.
How does a teacher explain how to solve it — how does a teacher effectively facilitate understanding?
When talking about this the other day, some people on our team were worried about one bad thing happening. I was worried about another.
Some Concerns.
CONCERN 1: A CLEAN MODEL … AND LITTLE ELSE.
My colleague Katie’s concern is that simply walking through a modelled example with a teacher writing this on the board -
2x + 5 = 15
15–5
2x = 10
x = 5
And then launching kids into independent practice….
….Will lead to lots of kids who don’t know how to get started on any problems. It’s the kind of thing that looks acceptable — look, a teacher’s teaching, and look, kids are now practicing! — but that’s ultimately fool’s gold. Sure, you ensure a lot of time for independent practice. But you haven’t set up kids properly, and so much of that time, for many of those children, is lost.
What about this, I wondered -
2x + 5 = 15
15–5 Subtract 5 to get the “x term” by itself.
2x = 10 Divide by 2 to Isolate x.
x = 5
i.e. Super simple qualitative descriptors that annotate what’s happening as you walk through the equation. A true worked example! (For people in to worked examples — and their surprisingly powerful showing in lab and classroom experiments — see overview here.) Maybe this would crack a “2” via the TEACH framework:
The content does seem at least “somewhat clear.” But my colleague Dorcas, who observes 20 lesson/week and gives feedback to teachers and our Design Team, reports that kids often focus more on copying the language/example, and LESS on thinking about what it actually means. And Katie’s concerns are borne out — kids stuck, unable to get going.
(Which brings us to a real tension on how to model academic language — do you make the language super technical and precise, or do you make it chattier, simpler, more informal, and thus plausibly more easily understood? Hard question that we’ll tackle on a different day.)
For now, let’s stipulate Katie’s point — that simply writing the model on the board (even with annotation) isn’t sufficient to the standard of an “explanation of content that is clear.”
Which brings us to …
CONCERN 2: TOO MUCH TALKING.
Easy for me to imagine LOTS of extraneous “teacher talk” on a problem like 2x + 5 = 15. I think walking through that problem should take no more than 3–5 minutes (max), if your class is really cooking, and there’s good, propulsive momentum, and you’re asking tightly aligned questions. Taking much longer than that means you’ve lost focus; you’ve lost the thread.
A reasonable devil’s advocate point of view would be that you can unlock misconceptions quite easily by asking kids lots of questions, and belabouring certain points, even on very “simple” examples.
I would bring you bring to Bold, et al.’s paper. That kind of hyper-efficient questioning is not common in any classrooms anywhere- it’s like a 90th percentile teacher move. Teachers who are super-skilled with pedagogical content knowledge can take lots of fun paths with this problem. They might do something like:
Teacher writes: 2x + 5 = 15
Teacher: “What do we do first?”
Student: “Add five to both sides.”
(Teacher takes kid at face value, adds 5 to both sides.)
Teacher writes:
2x + 5 = 15
+ 5 + 5
2x +10 = 20
Class: Some bewildered faces.
Teacher: “We added 5 to both sides. But we didn’t get closer to isolating “x.”But that’s ok!”
And on and on. The teacher uses this to make a point about algebraic equivalence — that adding “5” is not *wrong* — but that it does not get us closer to isolating for the unknown. Maybe takes a few informal class polls to figure out how to get the problem back on track. But then, crucially, gets back on track — really fast, like, within a minute. With a skilled teacher, this is a very powerful path; the brief detour a nice investment in building student understanding.
But it’s also a tightrope walk. I’ve been fortunate to observe some unbelievable teachers pull off organic conversations that take one tiny comment from a kid and spin it into a discussion that seems beautifully, almost eerily orchestrated — discussions that push on kids’ procedural and conceptual understanding.
But that’s not the common case. The common case is teachers talking way, way, way too long. And that’s very confusing! Engelmann writes:
“The mind is lawful. What humans learn is perfectly consistent with the input they receive…if there is more than one possible interpretation of what you’ve presented, some of your kids are going to pick up on the wrong one. The lower performing your kids are, the more often they’ll pick up on unintended interpretations.”
i.e. More often, the fancier you get, we’re left with a “1” via the framework…
…and not a “3.”
The teacher veering off topic, or over-explaining. Totally understandable, and very common … but not effective.
*
So we’ve established two things to stay away from — (i) a worked example with no modelled-thinking for the kids, (ii) an over baked example that takes too much time.
What’s the alternative?
A FRAMEWORK
Katie’s push is that 3 classroom “moves” (credit to Doug Lemov) can be used to great effect to maximise efficiency during the initial launch of the problem:
Maybe we’ll talk soon on about how, and when, to use these techniques, digging in more on 3.4 …
… or how to support teachers on “modelling their thinking” in a problem like “2x + 5 = 15.”
Sean Geraghty is Chief Academic Officer, Bridge International Academies. He used be a teacher so understands how important it is to feel empowered and supported. Now, he ensures that teachers and pupils have the best possible experience and achieve the best possible results by leading Bridge’s, “Learning Lab.”
