Attention Anti-RCT crusaders. Scaled up interventions can work too.
Over the years, we have seen anti-RCT crusaders on rise. The primary argument being ‘Development is messy. These interventions don’t work at scale. These aren’t silver bullets’.
This is frankly a lazy argument for several reasons.
One, it presumes that the goal of RCTs is to only find interventions that are to be scaled up which is blatantly false. RCTs serve various other purposes like testing popular hypothesis, generating evidence for cost effectiveness of approaches and so on.
The former set of studies like effect of increasing teacher salaries or using guest teachers dispel popular notions in media that low teacher salary is the problem and that increasing salaries will help. Using guest teachers has screwed up our system. They are the one’s mainly responsible for degradation of education system. Well, RCTs showed that it isn’t the case. In fact, in some cases, untrained teachers are more effective than regular teachers. It’s highly essential to dispel such hypothesis arising out of instincts because people cling on to these hypotheses refusing to see the picture beyond.
The latter set of studies, like the de-worming study provide evidence on cost effectiveness persuading governments to take up these interventions. It is well known that students have worms and that deworming tablets deworm students. The issue was lack of effort from governments to get this done. The deworming studies showed that with low cost one can both reduce worms and also increase school attendance, prompting governments to take action.
All such valuable contributions were overseen by the nitpickers.
Two, effectiveness of an intervention is a combination of ‘technical know-how’ and ‘capacity to implement’. Capacity to implement is further dependent on the complexity of ‘technical know-how’. In contexts where capacity to implement itself is low, the initiatives that are to be implemented have to be chosen carefully. The first step is to at least take up initiatives that have shown promise in contexts with strong intervention. It means that such interventions are good at ‘technical know-how’.
RCTs provide useful data on screening such initiatives that can be taken up. The important contribution of RCTs is that, given the given capacity of the government, it can tell you the intervention that can work. It often turns out to be one with simple technical know-how but it’s important to get at least these initial incremental gains.
The earlier interventions of scaling up of Pratham’s Teaching at the Right Level (TaRL) didn’t prove to be successful. Pratham and JPAL worked on it, adapted their approach and implemented in Haryana and Uttar Pradesh. The latest paper on this is out and says that the new scale ups have been successful.
In Haryana, teachers received support from government resource persons trained by Pratham, and implemented the approach during a dedicated hour. In Uttar Pradesh, Pratham volunteers implemented high-intensity, short-burst “learning camps” for 40 days, in school and during school hours, with additional 10-day summer camps. Both models proved effective, with gains in language of 0.15 standard deviation in Haryana, and 0.70 standard deviations in Uttar Pradesh, on all students enrolled in these schools at baseline. These two models provide blueprints that can be replicated inside other government systems
There’s no end to the pessimistic nitpickers in our world. Nitpicking and dismissing is the easiest thing to do. Evidence is the best possible effort to dislodge such priors.
Having said that, we shouldn’t get carried away by these results. TaRL is a relatively simple intervention with least complex technical know-how and address primitive needs of children (even those are missing in our system) like alphabets and numbers. One should note that current capacity capabilities may implement such simpler interventions but teaching higher order skills require higher order technical know-how for which we aren’t ready it. There’s lot to work on that front.
An analogy might help understand this. Our system is equivalent to a patient suffering from dysfunctional digestive system and isn’t able to digest anything. Over the years many foods (interventions) have been tried, often with false presumption that ‘food’ is the problem, glossing over lack of digestive capacity. Nothing worked.
Now, all foods are shunned and a simpler material, glucose is given to the person, and the person is able to digest it. It’s helping the person in short run. It doesn’t mean that the digestive capacity problems of the person are addressed. It only means that given the current capacity, this is the best food that can be digested. There’s lot more to be done. One can’t live on fluids through out the life. The capacity to digest complex food has to be developed.
In the above example, the simpler food glucose is TaRL. TaRL is an intervention that has proved to be working even in current capacity constraints, with appropriate approach.