The Standardized Test Blues

#2 in “The Good Schools Project”
← previous post

image courtesy of greatschools.org

After spending my evening crammed into a church basement and hearing I needed to spend 10+ hours sitting in circle time, I decided there had to be a shortcut to find my kid a good school.

Enter GreatSchools, a non-profit devoted to empowering parents by providing school information and rankings. I recently had the opportunity to interview Carrie Goux, GreatSchools’ Vice President of Communications. She explained that the site was specifically designed for parents. GS wants to provide the school information that parents value so that they can make the best school choice for their family and also utilize the site’s information to work to improve their schools.

Walking through the GreatSchools website, the first thing that pops out at you is the “summary ranking” — a colored circle with a ranking from 1–10. Schools ranked between 1–3 have student test scores below state averages, 4–7 are average, and 8–10 are above average. Each of these categories are colored red, yellow, or green respectively. Currently the summary ranking is comprised of test scores as compared against state averages, and, where available, (as in Washington state) student “growth,” which is determined by improved or diminished test scores year over year.

The website went through a recent revamp and now includes sections disaggregating achievement by race, disability, and socioeconomic status. At the bottom there is also space for parents to leave reviews. In the near future GreatSchools plans to incorporate more data into its summary rankings.

The GreatSchools website offers many helpful articles and collects school data in a very accessible platform. If parents are interested primarily in test scores, the site provides an easy way to compare schools.

But I’m of the opinion that standardized test scores fail to provide a complete picture of school quality. I’m disappointed that more data is not currently incorporated into GS’ summary ranking.

It is tempting to hold up standardized tests as an objective, scientific measurement of quality. Certainly they can provide us some data about student achievement. But there are also a lot of problems with standardized testing. For example, there are no provisions for English Language Learners (ELL) when it comes to taking a standardized English test. Think back to your high school Spanish class and imagine if you were asked to take the same Spanish test as a native speaker of that language. You would fail, and your poor score would dramatically impact the class average. Some Seattle schools have a high proportion of ELL students, so we should not expect those schools to score very well on standardized English tests. It doesn’t necessarily follow that kids aren’t getting a good English education at that school.

Looking past problems with averages, the tests themselves also present issues. Many tests are written in such a way that can trigger “stereotype threat” — an anxiety that you will confirm a negative stereotype about your group, such as “girls are bad at math” or “black people aren’t smart.” A 1995 study (which has since been repeated many times) showed that how a test is described to the people taking it can dramatically effect participants’ scores.

image courtesy of wikipedia.org

In this study, a test was described either as “diagnostic of intellectual ability” (the stereotype threat group) or “a laboratory problem-solving task that was nondiagnostic of ability” (the control group). You can see how much scores changed depending on how the same test was described. (For more information on stereotype threat, check out this article.)

These tests do not descend from heaven, perfect, empirical measures of student achievement. They are designed and tested by regular flawed, biased people. It’s important to question these methodologies and seek ways to eliminate bias in their results. We can’t take it for granted that they are perfect measurements, and I question how much we rely on them for both our own school choices and for how law makers apportion resources.

When I raised my concerns over test bias with Ms. Goux, she explained that GreatSchools receives information from the state and is not in a position to question the results of state testing. She did point out that the data on the site is disaggregated by groups and that parents can use that information to engage with school administrators and teachers over “achievement gaps” between groups.

I look forward to GreatSchools summary ranking revamp and hope that it will include data which can provide a more wholistic approach to school evaluation.

Leaving aside questions of standardized test scores, the GreatSchools site exposes one huge problem in Seattle Public Schools with stunning clarity.

Next time on “The Good Schools Project” Katharine spies an elephant in the room and decides to approach it, cautiously, perhaps with peanuts.

next post →