Why Do So Many School Districts Fail to Improve?
In his new book, “The Knowledge Capital of Nations”, Stanford’s Rick Hanushek concludes that in today’s digital economy, “long-run economic growth is overwhelmingly a function of the cognitive skills of the population.”
Due to rapidly accelerating technology and improving K12 performance in other nations, the real bar for “college and career readiness” is rising at an accelerating pace. If America’s K-12 school systems can’t keep up, our children are guaranteed to struggle, likely for years to come.
But that is exactly the problem we face today. A new bipartisan report from the National Conference of State Legislators put it bluntly: “Most state education systems are falling dangerously behind the world in a number of international comparisons and on our own National Assessment of Educational Progress, leaving the United States overwhelmingly underprepared to succeed in the 21st century economy” (“No Time to Lose”, August 2016).
In a previous article (“In K12 Education, Five Forces Are Driving Three Endgames”), I described why the current K12 status quo cannot survive, and proposed three possible scenarios for the future. The first is substantial improvement in the performance of traditional district-run schools. The second is aggressive expansion of charter schools and other forms of choice. And the third is a descent into fiscal chaos and increased social and political unrest, which some states are sadly approaching today.
In this article, I will present a case study that shows why the first scenario is so difficult in practice. To be sure, there are school districts — like Denver and Washington, DC — that have made major changes and produced substantial performance gains. Unfortunately, they are well-known because they are the exception, not the rule today in the United States.
Before delving into our case study, let me provide some brief background. I’m a business executive and political Independent, with almost forty years of experience around the world. As a banker, consultant, and corporate officer, I’ve spent most of my career improving the performance of troubled organizations. For the past fifteen years, I’ve devoted all my volunteer time to improving K12 performance, in New England, Alberta Canada, and for the past seven years in Colorado (where I have chaired the Accountability Committee at a district-run high school with 50% at-risk students, and also served on the District Accountability Committee).
And now, on to our case.
Jefferson County Colorado lies to the west of Denver, and has a population of almost 600,000. It is quite affluent — 42% of adult residents are college graduates, and median household income was $70,164 in 2015.
The Jefferson County School District is the 35th largest in the United States. It serves 86,000 students, with 160 schools and total revenues of about a billion dollars per year. Of Jeffco’s 4,900 teachers, 98% are rated by their school principals as effective or highly effective. One third of Jeffco students are eligible for free and reduced lunch (which includes students who “choice-in” to Jeffco from surrounding districts), and 11% attend charter schools.
Despite Jeffco’s affluence and spending on public schools, its academic results have been mediocre for the seven years since we moved here from Calgary.
On the 2016 state assessments (Colorado is a member of PARCC), only 42% of Jeffco 3rd graders met or exceeded the state standard for English Language Arts. For 6th graders, 49% met or exceeded the standard, while only 46% of 8th graders met or exceeded it.
On the 2016 state math assessment, 45% of 3rd graders, and 41% of 6th graders met the state standard. In 8th grade, about 1/3 of Jeffco students take an advanced math assessment (e.g., for algebra), while 2/3 take the assessment for grade level math. In the latter group, only 19% met or exceeded the state standard.
On the 2016 ACT, which is taken by every 11th grader in Colorado, only 32% of Jeffco students met all four college and career ready benchmarks (ACT results are highly correlated with scores on the ASVAB test that students who want to serve in the military must take, as well as pre-employment screening tests).
Finally, about one-third of Jeffco students who attend college must take one or more non-credit remedial courses.
Jeffco’s mediocre performance has many root causes.
Let’s start with the fundamental nature of the system that produces these results.
Like other large organizations, Jeffco is a complex adaptive system of systems, comprised of multiple agents who have a dense web of relationships with each other. These agents are constantly adapting their behaviors in light of their impact on the achievement of various goals, which themselves sometimes evolve over time. There are also many constraints on their actions, at the level of the school, the “articulation area” (a high school and its feeder schools), and the district.
In complex adaptive systems like Jeffco, the effects one observes often have multiple causes that are difficult to disentangle. Moreover, these emergent effects are often characterized by non-linearity and time delays. Collectively, all these factors make the behavior of complex adaptive systems extremely difficult to forecast, particularly as the time horizon lengthens.
How does one go about improving the performance of an organization like Jeffco? Experience has taught me that the search for silver bullet solutions based on accurate predictions of the future is fruitless. In a complex adaptive system, a leader’s only choice is to experiment and learn their way to better results — to evolve their way to survival and success, as it were.
A core challenge for such leaders is therefore to strengthen and accelerate the three basic drivers of all evolutionary processes: feedback, external selection, and internal adaptation.
Consciously or unconsciously, defenders of the status quo in Jeffco and more broadly in US K12 education seek to weaken all of these drivers.
Assuming the existence of selection pressures, feedback enables a system to deliberately improve its performance — in evolutionary terms, to reach a level of fitness that prevents it from being selected out of its environment (i.e., to fail). In the absence of feedback, survival results only from luck.
In K12 education, many critical feedback processes have been degraded or are under attack.
The College Board and the ACT Organization have noted problems with both grade inflation and a wide variation in the grading standards used across high schools (see, respectively, “Investigating Grade Inflation and Non-Equivalence” and “High School Grade Inflation from 2004 to 2011”). Contrast this with Alberta, where grading on a curve was eliminated in order to strengthen the validity and power of teacher assigned grades as a feedback process.
To cite one very painful example of the impact of weakened feedback from teacher-assigned grades, our Jeffco high school uses the NWEA Measures of Academic Progress to assess the reading and math skills of incoming 9th graders. In recent years, we have found that 25% of these students were reading between a 7th and 5th grade level, and another 25% were at 4th grade or below. Yet when we looked at these students’ transcripts, we saw Bs and Cs and steady promotion from grade to grade.
Beyond problems with teacher-assigned grades, attacks on standardized assessments, by both the teachers unions and opt-out movement, are also very well-known. A common refrain is that students are “more than a score” and there is more to college and career readiness than the academic competencies we measure today.
I have two responses to these claims. First, while reading, writing, and math are undoubtedly not sufficient measures of college and career readiness, surely they are necessary ones. Indeed, it is impossible to claim a student is career and college ready if they do not possess these critical competencies.
Second, we do not lack ways to assess non-cognitive skills and traits that are key components of college and career readiness. For example at our high school we have very successfully piloted a set of assessments from Pairin (www.pairin.com) that have helped us to assess and improve students’ non-cognitive skills.
Last but certainly not least is the nation-wide attack on teacher performance feedback measures and processes. Research has shown that one of the key motivators of professionals is the desire for mastery; how teachers are to achieve this in the absence of objective, measurable feedback is a question their unions have yet to answer.
Any process that enables parents and students to choose the school they will attend puts performance pressure on every school in the system. If enough students leave a school, teachers get laid off and the school itself may close.
Jeffco has multiple forms of school choice. Within the district, parents can choose between district-run schools. In Colorado, parents can choose schools across district lines. Parents can also choose to attend charter schools. And if they pay more, they can attend private schools of various types.
As more and more parents have elected to send their children to charter schools, it has come as no surprise that attacks on them by the teachers unions and their allies have increased. To the extent that these attacks are successful, they will reduce selection pressures in the system, and thus weaken another critical evolutionary driver of performance improvement.
For the remainder of this article, I will take a closer look at internal adaptation, which is the third evolutionary driver. Since moving to Jeffco from Calgary seven years ago, I have seen how poor management, toxic culture, and weak governance have combined to minimize internal adaptation and block improvement in the district’s mediocre results.
It is well established that superior management can have a substantial positive impact on organizational performance (e.g., see “Management as a Technology?” by Bloom et al). In Jeffco, multiple management weaknesses exist that have profound negative effects on performance. I will offer just a few telling examples.
When it comes to efficiency, Jeffco is a classic example of an organization that focuses far too much on financial accounting, and far too little on understanding its true economic costs. The discipline of activity based costing — which connects goals to the activities required to achieve them, and those activities to operating and capital costs — is completely absent in Jeffco.
Let me offer a painful, but very telling example. On the district’s “financial transparency” website, you can access individual Jeffco purchase transactions. But what you cannot do is aggregate them into meaningful cost categories. I have repeatedly asked district leaders how much Jeffco spends on teacher professional development, (including both direct costs and the value of employees’ time), what metrics are used to measure its effectiveness, and what return the taxpayers are getting on this investment. I have never received an answer.
In their 2015 study “The Mirage”, The New Teacher Project found that other large districts like Jeffco spend (in direct costs and the value of employees’ time) nearly $18,000 per year per teacher on professional development, with no positive return. Given that Jeffco employs about 4,900 teachers, its annual economic cost for PD could be more than $88 million — almost ten percent of its billion-dollar budget. And the district’s poor student achievement scores suggest it isn’t producing a positive impact here either.
In the absence of this type of rigorous cost analysis, it is no surprise that Jeffco, like so many school districts across the country, is constantly telling taxpayers that it needs more money, even though it cannot provide any meaningful insight into how efficiently the billion taxpayer dollars it already receives are being spent. Unsurprisingly, local voters have been very reluctant to support higher school taxes.
Let me now turn to effectiveness. In 2009, the Colorado Department of Education mandated the use, at both the school and district level, of a standard continuous improvement process.
While it goes by many names (e.g., Deming, Kaizen, PDSA, etc.), in Colorado K12, it is called “Unified Improvement Planning.” The UIP process involves four broad steps: (1) Diagnose and prioritize the root causes of student achievement shortfalls versus goals; (2) Identify, assess, and select improvement initiatives to address priority root causes; (3) Implement these initiatives; and (4) Check to see what impact they have had and identify lessons to learn.
While this process has for at least 25 years produced substantial performance gains across multiple organizations and sectors of the global economy, it has singularly failed to do so in Jeffco (and, more broadly, in Colorado K12). Having accumulated seven years of data, it is not hard for me to explain why this is the case — every stage of the UIP process is broken.
Many potentially important root causes are off limits and cannot be discussed (e.g., teachers with high absence rates, weak curriculum, etc.).
Identification and selection of improvement initiatives is rarely guided by evidence from external research studies, and is most often driven by instinct, anecdote, ideology, and politics.
Implementation is often poor, hardly ever tracked, and never aggregated across 160 schools, much less reported to the public.
Improvement initiatives’ results are rarely rigorously evaluated, and almost never compared to the results one would have expected to see based on research findings, and, again, never reported to the public.
In sum, Jeffco’s implementation of the state UIP process usually amounts to a time wasting Kabuki dance — an exercise in form-filling and box-ticking for the regulators, without any real substance or measurable positive impact on student achievement results.
The second obstacle to internal adaptation in Jeffco is the district’s toxic culture (a term I do not use lightly).
From research in the rapidly expanding discipline of social network theory, we now know that a relatively small fraction of committed agents in a population (about 10% or more) can produce and sustain a significant cultural shift (e.g., see “Social Consensus Through the Influence of Committed Minorities” by Xie et al, and “Committed Activists and the Reshaping of the Status Quo Social Consensus” by Mistry et al).
We also know from three independent studies that the fraction of ineffective teachers in a given population is likely to be between 13% and 15%.
In 2014 the Harvard/Education Next Poll asked teachers to anonymously grade other teachers in their local schools. Teachers gave 13% of their peers a D or an F.
In 2015, the Gallup Organization found that 13% of the nation’s teachers were what it termed “Actively Disengaged”, which is defined as follows:
“Actively disengaged people operate from the mindset, ‘I’m okay. You’re not okay.’ They believe that they’re doing what needs to done, and everyone else is wrong. Negativity is like a blood clot, and actively disengaged employees sometimes clot together in groups that support and reinforce their beliefs…”
“Actively disengaged employees also may close themselves off from anyone who will challenge them to become part of the solution, rather than staying part of the problem. This is key to understanding the difference between an engaged and actively disengaged person. An engaged person occasionally becomes negative. We all do. But an actively disengaged person finds it almost impossible to become part of the solution, because they thrive on being part of the problem” (“Lack of Teacher Engagement Linked to 2.3 Million Missed Workdays”, January 2015).
This year, Grissom and Loeb published “Assessing Principals’ Assessments: Subjective Evaluations of Teacher Effectiveness in Low- and High-Stakes Environments.” In private assessments, principals rated 15% of their teachers as ineffective (in official assessments, the percentage rated ineffective was much lower).
Combining the social network and teacher quality findings, it is easy to see how a toxic culture can develop in a school or district.
What sustains such cultures over time is the combination of teacher pension plans whose benefits are heavily back-end loaded (creating a strong financial incentive to stay in teaching for 25 or 30 years), along with union contracts and state laws that make it practically impossible to fire the 13% to 15% of teachers who are ineffective and often actively disengaged from their work.
It is hard to overstate the multiple negative effects this has on a school district.
For example, if a principal cannot fire weak performers, she has little incentive to give them poor performance ratings. Unsurprisingly, 98% of Jeffco’s 4,900 teachers are rated either effective or highly effective, despite the fact that only 32% of the district’s 11th graders meet all four ACT college and career ready benchmarks. This makes voters cynical and teachers overconfident.
A compensation system that (as is the case in other professions) provides superior rewards to the best performing teachers would also call attention to the weak performers who cannot be fired. It is therefore not surprising that teachers’ unions aggressively support compensation systems that are based on seniority and the accumulation of credentials which research has shown have no positive impact on student achievement.
To be sure, I have met many teachers who would gladly accept differential compensation based on their (or their school’s) performance. But they are clearly a minority (or perhaps a silent majority) in union-dominated districts like Jeffco. And so our seniority and credentials-based compensation system continues to demoralize and drive out our best teachers while unfairly rewarding our worst.
Another negative effect of Jeffco’s toxic culture is a high rate of teacher absenteeism. Based on the latest data from the federal Department of Education’s Office of Civil Rights, 30% of Jeffco teachers use more than 10 personal and sick days each year — and 10 days is the point at which OCR’s research says the impact of teacher absence on student achievement becomes significant (to say nothing of its budgetary impact).
To put this in perspective, a recent study found that nationally, 27% of teachers were “chronically absent” based on this same 10 day standard. A subsequent study examined a range of potential (and reasonable) explanations for this high absence rate, and concluded that 17% of these absences could not be explained.
Jeffco’s toxic K12 culture also manifests itself in widespread aversion to quantitative analysis of results (which only highlights poor performance), as well as the district’s resistance to experimenting with new approaches (e.g., competency rather than grade-based learning), especially when they could pose a threat to teacher jobs or require significant change in teacher knowledge and skills (e.g., personalized learning that leverages technology).
The resistance to change engendered by a toxic culture results in a strong organizational norm to be “collaborative” — with any hint of conflict indicating that you have violated it and should be punished. Unfortunately, substantial performance improvement almost always requires substantial change, which inevitably generates substantial conflict. Time and time again I have seen potential K12 change agents in Jeffco undermined by its “you must be collaborative” norm.
This is also why school districts like Jeffco cannot attract and retain leaders from other sectors of the economy, such as business and the military, for whom the imperative of superior performance usually takes precedence over the desire to avoid inter-personal conflict.
I have also seen the collaboration norm inhibit closer cooperation between K12 and outside organizations, for example to take a more integrated and coordinated approach to meeting the challenges faced by at-risk students and their families (such as Massachusetts’ “Wrap Around Zones” initiative). K12 leaders usually react very badly when people from other organizations tell them they need to change; consequently these initiatives tend to peter out quickly, with little impact.
To be sure, I have seen district-run schools where individual mastery and collective excellence are relentlessly pursued by a professional administrative and teaching staff. But they are all too rare, and often succeed in spite of, not because of their district. And more often than not, these “tall poppy” schools eventually find themselves under attack by resentful peers, often with the connivance of the district head office.
A final manifestation of Jeffco’s toxic culture is its insularity, which, as is the case with any organization, inhibits its ability to adapt, particularly in a rapidly changing environment. There are structural features of the Jeffco organization that are in part intended to stimulate contact between district personnel and people outside of K12. However, from the PTA to school and district accountability committees (which were established by the state legislature as part of the state school accountability law), Jeffco has repeatedly sought to fill them with “district cheerleaders” who can be counted on to communicate the district’s desired narrative, and not to challenge it.
In sum, Peter Drucker’s famous observation — that “culture eats strategy for breakfast” — perfectly describes what is perhaps the most important and under-appreciated obstacle to substantial performance improvement in many US school districts.
Like many districts in the United States, Jeffco is governed by a locally elected school board. In local elections, board members often derive substantial support from the teachers’ union, reminding one of the famous quote by New York City municipal union leader Victor Gotbaum, who forty years ago observed that, “We have the ability, in a sense, to elect our own boss.”
As a result of this morally corrupt dynamic, few local school boards are willing to aggressively challenge either management or the teachers union, and instead see their primary role as being cheerleaders for their district.
This is certainly the case in Jeffco, where management and board members expend great effort spinning the district’s achievement results and trying to convince the public that hamburger is steak.
In addition, as I have written elsewhere, many school districts lack high quality strategies (see, “Does Your School District Have a Real Strategy?”). Combined with the absence of activity based cost management systems, this predictably leads to an ineffective resource allocation process, with next year’s budget essentially equal to last year’s with a few changes around the edges. Put differently, weak management and governance have enabled the annual school district budgeting process to become a key protector of the failed K12 status quo.
Strategic risk governance is also usually absent; I have never seen a school board ask a management team to identify the major risks to the implementation of its strategy, apart from the routine complaint that the district needs more money. Indeed, far too many people in K12 fail to grasp the essential idea that strategy is about how to survive and thrive with limited resources in the face of uncertainty.
Finally school boards’ monitoring of district performance is usually incredibly weak in comparison with what one regularly observes on private sector boards. As noted above, cheerleading by board members is far more common than skepticism and tough questions.
When we moved to Jeffco, I didn’t set out to write such a depressing case study. I was actually quite hopeful about the chances for substantially improving the performance of district-run schools, having come from the Province of Alberta where I had already seen that happen.
Sadly, after seven years of struggle I have concluded that our Alberta experience will not be repeated in Jeffco, or in Colorado. The interlocking and reinforcing obstacles of poor management, toxic culture, and weak governance have proven impossible to overcome in too many school districts.
In the private sector, a billion dollar organization with Jeffco’s many problems and mediocre performance would soon go bankrupt or be taken over. Evolutionary forces would drive creative destruction and produce higher performance.
Unfortunately, that doesn’t happen to public school districts, which each year waste billions of dollars while condemning tens of thousands more children to a lifetime of struggle, and our nation to a future of weak economic growth, increasing poverty, and growing social and political conflict.
As John F. Kennedy once said, “Our progress as a nation can be no swifter than our progress in education. The human mind is our fundamental resource.”
President Kennedy’s words have never been more true.
Tom Coyne is a co-founder of K12 Accountability Inc. (www.k12accountability.org)