Who Rates Teachers This Way?
In his recent “State of the State” address, New York State Governor Andrew Cuomo proposed substantial changes to the Annual Professional Performance Review (APPR) scoring system by which teachers in the state are rated every year. Currently, NYS teachers are rated under a system where classroom observations count for 60% of the rating, 20% is a state-provided score based on student performance on standardized exams (for subjects where exams exist), and 20% is a locally-determined score based on student growth. Based on these criteria, teachers are assigned one of four ratings ranging from “Highly Effective” to “Effective”, “Developing”, and “Ineffective.” Teachers who rate as either of the bottom two categories are placed on a Teacher Improvement Plan (TIP). Rating “Ineffective” for two successive years can be grounds for termination.
The Governor’s proposed system would make the following changes:
● The 20% local growth component would be eliminated.
● APPR ratings would be based on 50% classroom observations, and 50% state-provided scores based on student exam performance.
● The weighting of the two categories would be adjusted so that if a teacher scores as “developing” or “ineffective” in the exam-score category, they can not score higher than “developing” on the overall evaluation, regardless of the classroom observation portion of the APPR.
In making these proposals, the Governor is proposing a system that actually goes beyond what New York State Board of Regents Chancellor Meryl Tisch suggested in her December 2014 letter to Cuomo’s office (NYSED, 2014).
Assuming that the goal of these proposals is to increase the quality of education in New York State, it seems reasonable to expect that systems like the one that the Governor has proposed are widely used in educational systems that we would like to emulate in the State. Unfortunately (if not unsurprisingly), that does not seem to be the case. We are hard-pressed to find any model educational system that uses a teacher rating system like the one that the Governor has proposed. Outside of all the other issues related to the Governor’s proposals, this one—the lack of evidentiary support for his new APPR model—should be of primary concern to anyone who is interested the long-term prospects of New York State public education.
Teacher Evaluations in Other Countries
It is not difficult to learn how other countries evaluate their teachers. Before considering state-level evaluations, let’s take a journey through some of the major educational players on the international stage.
For a variety of reasons, the educational system in Finland is often viewed as one of the best in the world. As arguable as that claim might be, it’s easy to look at what the school system of Finland does with regard to teacher evaluation. There’s only one problem with this approach: Finland doesn’t have a centralized teacher-evaluation system (OECD, 2013). Finland’s teachers are evaluated entirely by locally-determined measures established through agreement between schools and the teachers’ union (there are also no standardized exams in Finland’s schools). This fact stands to demonstrate that outside of what the Cuomo (and Obama) administration may suggest, centralized teacher-ratings are not a requisite for an educational system that is viewed as a model for the World.
Even if we don't want to emulate Finland’s approach, we can still take instruction from the way other nations utilize student exam scores within the context of teacher evaluation. Much has been made of the utility and limitations of using international exam scores to conclude too much about the strength of a particular country’s educational system. That noted, we can still use these exams to help focus in on countries who have educational systems that we may want to emulate in America. At the very least, this seems to be the kind of data that is most important to folks who think like Governor Cuomo does about education: As he made clear in his Address, Test scores are to be the coin of the realm. So for the sake of argument let’s look at how the top 5 scoring countries on the 2011 Trends in International Mathematics and Science Study (TIMSS) Mathematics assessment go about the process of evaluating their teachers (TIMSS, 2012):
● Singapore (#1): Teachers are evaluated using a complex process that looks at 16 different areas to determine teacher effectiveness (NCEE, 2014)
● Hong Kong (#2): Teachers are not centrally evaluated (NCEE, 2012)
● South Korea (#3): Teachers are not centrally evaluated outside of their schools (Ripley [WSJ], 2013)
● China (#4): Teachers are not centrally evaluated (NCEE, 2012)
● Japan (#5): Teachers are not centrally evaluated (NCEE, 2012)
Looking at these five countries, it’s obvious that Finland is not some sort of bizarre outlier. None of the countries listed above utilize test scores in the kind of over-emphasized, zero-sum role that Governor Cuomo has proposed for New York State’s teacher evaluations, and none of them view them as centrally important in understanding the picture of a teacher’s effectiveness in the classroom. Given the absence of centralized evaluation of teachers from the evaluative frameworks of these countries, one may be left to wonder if a system like the Governor’s is something to strive for at all. Simply put, if test-score-centric teacher evaluations are so useful, why aren’t they being more widely used?
Teacher Evaluations in Other States
Being that none of the 6 “leading” nations that we have examined use test scores in the way that Governor Cuomo has proposed, we might wonder if there is something unique to America that requires high-quality education to utilize extreme weighting of test-scores in teacher evaluations. We can easily examine this claim as well, assuming we are willing to again simplify our criteria to simplistically look at one of the variety of rank-order lists of states with regard to education. Choosing one such recent analysis (more or less at random), let’s look at the five “best value” states for education: New Jersey, Massachusetts, Vermont, New Hampshire, and Kansas (Klein, 2014). As above, it is not difficult to determine the role that exam scores play in the evaluations of teachers in each of these states:
● New Jersey: Exam Scores factor in to a maximum of 30% of a teacher’s rating in the 2014–2015 school year for the purpose of continued employment (AchieveNJ, 2013).
● Massachusetts: Exam scores do not factor in to a teacher’s rating for the purpose of continued employment (MassTeacher.org, 2014).
● Vermont: There is currently no uniform policy for using test scores to determine teacher ratings for the purpose of continued employment (Strauss, 2014).
● New Hampshire: There is currently no uniform policy for using test scores to determine teacher ratings for the purpose of continued employment (Evans-Brown, 2013).
● Kansas: There is currently no uniform policy for using test scores to determine teacher ratings for the purpose of continued employment (Hancock, 2014).
This picture established, we have to dismiss the notion that a tremendous valuation of exam scores on teacher ratings is needed at the state level to develop a high-quality teacher corps. Indeed, most of the members of the “top 5" education states are trying to diminish the effect of exam scores on teacher ratings, seeking waivers from federal requirements, and issuing state-wide cautions against overuse of test-score data. Looking at the larger picture of how states evaluate their teachers, this author was only able to find Louisianna using a similar system to the one that Governor Cuomo has proposed (Schneider, 2015). We should note that the same study that ranked the “top 5” states above, ranks Louisiana as 48th out of the 51 systems considered (update: a colleague in Georgia wrote in to tell me that Georgia uses a similar system. Georgia is 32nd on this particular list). Is this really the vision that we want to have for New York State’s schools?
The Broad Concerns with Exam-Scores in Teacher Evaluations
Unlike the lack of evidence supporting Governor Cuomo’s revised teacher evaluations, there is a large body of expert opinions that warn against such a system. Here is a quick sampling of some of the more easily-found examples:
● Both the American Statistical Association (Chetty, 2014) and the American Education Research Association / National Academy of Education (Stanford, 2013) have published detailed statements against the overuse of “Value-Added Models” of determining teacher effectiveness through student exam scores.
● The Gates Foundation has published the findings of their three-year Measures of Effective Teaching (MET) study, indicating that “Multiple measures also produce more consistent ratings than student achievement measures alone. (MET, 2013)”
● The Association for Supervision and Curriculum Development has published a position statement outlining the problems with the use of test scores in evaluations and calling for a 2-year moratorium on such evaluative usage (ASCD, 2015).
● Seven recent NYS Teacher-of-the-Year award winners have written an open letter to Governor Cuomo explaining the damage that his APPR plan would have on the profession (Dreher et. al, 2015).
What is perhaps most surprising is just how broad the consensus is against the type of use of exam scores that Governor Cuomo is pursuing. Frankly, we might wonder why he, as the Chief Executive of the State, is so strongly in favor of an approach to teacher evaluation that is so at-odds with the positions of so many experts in so many different fields. Even if he is not the chief architect of his education proposals, is it too much to ask that he function in his role to make those whom he has tasked with developing such a plan explain why they have concocted something that seems to contravene common sense?
We have to ask “why?”
Looking at the breadth of objection to the type of rating system that the Governor is proposing, and its absence from model educational systems at the national and international levels, one can only wonder why Governor Cuomo is pursuing such a policy. Attempts to make sense of these initiatives don’t lead to flattering conclusions: Either he is ignorant of the consensus that advocates against test-score centric teacher evaluation models, or he has decided that he knows better than a broad consensus of educators, researchers, and the entire educational systems of “high achieving” countries and states. We are not sure which of these possibilities is more troubling with regard to how the Governor thinks about the public education system of the state.
Given this analysis, it is clear that anyone who is actually concerned with the long-term health of the New York State public education system should be vocally, and stridently opposed to the education goals of its current Governor. This is not a partisan issue, or one that seeks to unfairly protect the jobs of the NYS teacher corps. There are ways to propose teacher evaluation systems that are in agreement with research and based on evidence from what is working in other places. This is not what the Governor has chosen to do. Rather than seeking to have a conversation with educators, students, parents, and all of the other stake-holders who value education in New York State, the Governor has chosen to propose an unsupported evaluation system with no track record of success in doing what he claims to want to do. And rather than attempt to build consensus on his proposals, Governor Cuomo has taken the position that he is not interested in perspectives other than his own on this issue. He is so strongly in favor of his education proposals that he is withholding state aid figures from districts until he understands just how eager the legislature is to support him in driving his education plan through without debate. It is difficult to believe that someone so vocally concerned with the future of NYS education would be willing to threaten the aid that districts need to provide for their most underserved student populations. It is similarly difficult to understand why he stands in opposition to reality itself on the matter of creating an effective teacher evaluation system. New York State residents should be very concerned about what their Governor seeks to do. We deserve better, and so do our children.
This article was written as part of the New York Teachers’ Letter project.