Charter Schools Have Only Achieved a Local Maximum in Student Outcomes

Over the last few days, I’ve been listening to season 2 of the excellent podcast StartUp, which chronicles the founders of online matchmaking service Dating Ring as they launch their company. While participating in the famed startup incubator Y Combinator, the founders faced a classic Silicon Valley decision — pivot or persevere — and this conundrum has a lot of relevance to education, particularly “no excuses” charter networks.

Dating Ring was founded on the concept of group dates — that meeting people in a group is less pressure and more conducive to finding the right person for you. This idea had taken the founders all the way to YC (no mean feat) and modest profitability, but they had begun to see their user growth and revenue stagnate. Should Dating Ring pivot to a new business model or persevere with the group dating concept?

The evidence is quite convincing that, on the whole, public charter schools achieve superior academic outcomes over traditional public schools. A 2013 study from CREDO at Stanford found that nationally, charters outperform traditional public schools in reading and tie in math, and a 2011 meta-analysis from University of California, San Diego found that charters get better results in a variety of areas, including elementary reading and math, middle school math, and urban high school reading (Separating Fact & Fiction: What You Need to Know About Charter Schools, pg. 8). If you restrict the analysis to looking just at no excuses charter networks, like KIPP, Achievement First, and Uncommon, the results are even more positive (No Excuses Charter Schools: A Meta-Analysis of the Experimental Evidence on Student Achievement).

All of these data would suggest that these schools are on the right track, and the major problem is the need to scale this model even further. However, a look at the actual students in these schools tells a very different story. Here are students lined up at Harlem Success Academy 4, courtesy of the New York Times:

The Harlem Success network gets outstanding New York state test scores, with 94% of students passing the math test compared to 35% statewide. But the eyes of these children tell a very different story: broken down and resigned to their fate, to spend their days SLANTing and following directions and working their butts off to pass a paper test. Stepping out of line, either literally or figuratively, will be punished severely.

While I may be reading a little too much into one photo, there is a growing realization among charter school leaders that “no excuses” may be at its limits. In Building a Better Teacher, Elizabeth Green devotes several chapters to charter school discipline and writes approvingly of those educators who realized that the model can go too far. And Katherine Reynolds Lewis just published an excellent piece in Mother Jones on how negative consequences for kids just make bad behavior worse.

But from the data alone, it would seem that the clear choice is to persevere, perhaps with some tinkering to alleviate the greatest excesses of charter school discipline. It makes total sense that an Eva Moskowitz of Harlem Success would stay the course. You’re already killing it in math, and given that, upping the reading passing rate from 64% to 94% seems like a totally solvable problem in the current system.

The issue is that we don’t have the right data. Test scores obviously measure something, and that something — literacy, numeracy, and depending on the test, critical thinking — can be quite important. But they don’t measure other crucial things: social-emotional learning, growth mindset, the ability to set goals and plan for the long term, curiosity, passion for improving the world. Nor do they measure how satisfied students are with the education they are getting or how satisfied teachers are with what they are being asked to deliver.

Some charter school networks are taking a different tack. KIPP’s “Six Essential Questions” measure critical outcomes, such as whether students stay in the network, teachers stay in the network, and alumni are equipped to complete college. Similarly, when Summit Public Schools calculated the college graduation rates of their first wave of alumni and were disappointed in the results, they totally reimagined their school model.

In the end, charter school networks that collect a wide range of data and consider them all carefully will almost certainly find that the no excuses model as it existed historically is merely a local maximum in student achievement. You can iterate on it a bit and probably push your college graduation rate up a few points, but it is going to take questioning fundamental assumptions about your network to achieve truly gap-closing outcomes. In other words, you need to pivot.

Indeed, this is exactly what Dating Ring did. When faced with their mediocre growth rate, they decided to get more data by surveying their existing users whether they preferred group dates — Dating Ring’s core innovation — or one-on-one dates. Users overwhelmingly preferred one-on-one dates, and the founders changed strategies, successfully raising a seed round of funding in the process. Dating Ring just had to find the right data.