Reflections on a Decade-Long Quest to Make Education Evidence-Based
Today is a milestone in a decade-long quest to learn how to best support students and teachers in our nation’s middle and high schools most affected by concentrated, inter-generational poverty. The early impact results from MDRC’s on-going, independent evaluation and randomized field trial of our Diplomas Now model, which is part of the U.S. Department of Education’s Investing In Innovation (i3) program, are out. They show for the first time with large-scale experimental evidence that it is possible to reduce the number of students who end sixth and ninth grade off-track to high school graduation and on the path to dropping out. Moreover, it is possible to do this consistently, across multiple districts in the nation’s highest-need middle and high schools.
My reactions to this are many.
Gratitude and relief. The intense efforts of teachers, principals, student support staff, our Talent Development Secondary school transformation facilitators, City Year’s AmeriCorps members, Communities In Schools site coordinators and others paid off. Thousands of students, who absent effective intervention would be on the path to dropping out and likely diminished lives, were instead given the support they needed to succeed in school.
Satisfaction that we have been able to complete a long arc from research to meaningful changes in educational practice. This arc stretches years: first from recognizing that our Talent Development whole-school reform models could help high-need secondary schools move from really struggling to decent but still left too many students behind to the first eureka moment when we saw that we could accurately identify, as early as sixth grade, those students who, absent effective interventions, were highly likely to drop out to the despair we felt when we ran the numbers and found that the most challenged schools often had hundreds of students with early-warning indicators, which overwhelmed their current capacity to intervene.
From that low point, we hunted for a solution and recognized we needed to solve both a scale and intensity challenge. That led to the excitement of finding our two partners: City Year, which, by providing a team of 10 to 15 recent college graduates doing a year of national service, offered a way to reach more than 100 students a day with additional academic and socio-emotional support and Communities In Schools with its integrated student support model and site coordinators who could provide case management to students facing significant challenges at home. We added a school transformation facilitator from Talent Development Secondary to be principals’ right-hand point person and the grease and glue who made the whole early warning and intervention system work. Together, this became Diplomas Now, which combines whole-school reform strategies with enhanced student support at the scale and intensity needed guided by an early warning system.
Then at lightning speed, we went from a single pilot middle school in Philadelphia to rapid expansion across 10 schools in New Orleans, Seattle, and Philadelphia, thanks to a big bet from the PepsiCo Foundation, to winning the largest i3 validation award during its inaugural year in 2010. This then led us to the most expansive randomized control trial ever done in secondary education in the United States — involving 11 major urban districts, 62 middle and high schools, and more than 40,000 students.
Five years later, we are today releasing the early impact results. The full study has another year to go as we follow students from sixth grade through middle school to see how they will fare in ninth grade, and ninth-graders all the way through to high school graduation.
I also feel befuddlement. In a nation where medical procedures are changed based on hundreds of randomized studies, and Google and Walmart conduct thousands in a year, such studies within education are still rare. The Diplomas Now study should be commonplace, rather than a historical accident brought on by a great recession and its stimulus response. In all likelihood, I will not be a part of another randomized field trial of this size and scope in my lifetime. That is problematic. We are learning a lot through this seven-year study, but when it’s done, many questions and tantalizing leads likely will go unstudied. To make education an evidence-based practice, we need to continually gather evidence at a large enough scale to solve challenging problems. By now, it should have sunk in: reliably providing children who live in poverty a pathway to adult success through a solid education is not simple or easy. We need to build an evidence base on how to do this more — not less — intensely than Walmart figures out product placement. At a minimum, we should spend one penny of every federal dollar invested in K-12 schooling on evidence gathering. That would total $500 million per year. And yet, the replacement program for i3 is budgeted at under $100 million.
Hope and frustration. We have made enormous progress in raising the national graduation rate by nearly 10 percentage points in the last decade and in reducing the number of low graduation rate high schools by half. Yet there are still a thousand high schools where graduation is not the norm, where chronic absenteeism and suspension rates are often as high as 40 percent or higher. These schools are primarily attended by high-poverty students of color. Each of these high schools likely has at least two feeder middle schools where large percentages of students fall far off track to graduation even before they spend a day in high school. We now are in the position to engage in insider trading for the social good. We know which schools will produce the next generation of dropouts. We know which students in these schools won’t make it without effective interventions. And we now have evidence that we know how to help them. Now, it’s just a question of will and social organization. Will we respond to students in need, waving their hands for help? We no longer can say that we need to gather more evidence before we know what to do. We have the answer now.
