The Semantics of Scores: Evaluating Our Relationship with NAEP Assessment Data
Edward Montalvo, Senior Education Writer
Many educators and parents panicked about the latest results on the National Assessment of Educational Progress (NAEP) when we saw drops across the board for fourth and eighth graders in math and reading. While alarm about the results was warranted, we knew there would be impacts from the pandemic. However, the degree of concern was not well-moderated.
Our response to high-stakes assessments has to be more prudent. Children enter school learning how to trace a marker around their hands and, within 20 years, learn to use that same hand for surgery, coding, or sculpting. A person could spend a quarter of their life in a classroom, which means we have to apply more thoughtful approaches when understanding the nuance and long-term conditions of student learning. We cannot afford to rush into short-term solutions.
I spoke with Stephen Owens, a senior policy analyst at the Georgia Budget & Policy Institute, about how we should respond to high-stakes testing such as NAEP — plus the SAT and ACT. Owens has written extensively on data, funding, and policy implementation in practice. I agreed with him on this main point: “No teacher was waiting on these test scores to see if something had gone wrong, and no one was surprised by the outcome.”
Owens is concerned that incomplete interpretations of testing data can lead to decisions that negatively impact students. “Oftentimes, these scores are used as a way to punish schools, especially schools educating more students of color and more low-income children,” he explained. Data from high-stakes assessments, which can only provide a snapshot of student learning, are often utilized as accountability measures. Research shows poor student performance can result in strict oversight, a narrowing of curriculum, and funding setbacks.
To rethink high school, we must better understand the intersections of data and school design. How we respond to new sets of information is just as important as the data itself.
What NAEP Confirms
NAEP is described as the Nation’s Report Card, providing us a snapshot of where student learning may be. But as Owens explained, we have to contextualize assessment scores and view them with our long-term educational goals when interpreting the data appropriately. “Decades of education research has shown that our schools sit downwind of the home and the neighborhood and the culture at large,” he said.
While the drop in math scores for both grades is the largest since the test began in 1990, that’s a percentage drop and doesn’t mean the actual scale scores are the lowest since then. Fourth-grade reading scores also fell the most since 1990. But current fourth and eighth-grade scores for reading and math are hovering at levels similar to 2004. “Many people are powering the United States economy right now [were] educated in the late nineties and early 2000s,” said Owens. The rate of the drop is what stirred such alarm.
Given the pandemic conditions, many believed the loss would have been worse. However, average scores in reading and math for fourth- and eighth-grade students for the long-term trend assessment have declined slowly since 2012. While we should be concerned about the impacts of COVID-19, we should be assessing further back as we plan for the future.
“It is a big drop from a nation not used to seeing any drops in NAEP scores,” said Owens. “But at some point, we have to recognize that our fourth graders in 2022 are not different wholesale creatures than fourth graders in 1992. And to expect something so qualitatively different out of their educational output is going to come with a cost,” he added.
Accelerations in technology and the rollout of Common Core State Standards (CCSS) in 2010 should have resulted in positive gains. However, when examining the trendlines from NAEP, it seems as if we were still recovering from massive budget cuts following the Great Recession in 2008. These cuts have not returned to pre-recession levels in most states. Coupled with the intense pushback to CCSS and teacher burnout, scores on the NAEP began a downward trend in 2012, followed by a plateau.
After the 2022 scores were released, a discourse of hysteria from news outlets, policymakers, and education leaders steered the conversation in the wrong direction. It’s not that the entire public education system is broken; rather, it’s flawed in vulnerable areas. Students of color, those who live in economically underserved communities, and our English language learners experience the brunt of public education’s failures after a decade of neglect. But it’s in these communities where stronger investments can result in more significant gains.
Our Relationship with Testing Data
High-stakes testing is like an X-ray on a broken arm — it confirms what we already know. Investing in our students and supporting the expertise of our teachers is the first proper response in galvanizing the efforts we need to promote academic recovery.
As Allison Rose Socol, the Education Trust’s vice president of P-12 policy, wrote, “Now is the time for greater investments in proven strategies that accelerate students’ learning and that address the persistent inequities underscored in those numbers.” Socol identified several strategies which help to support student learning, including a strong and diverse teacher workforce, a positive school culture, and authentic learning experiences. All of these align with XQ’s Design Principles for creating more exciting, rigorous teaching and learning in high schools.
Educators and advocates have been vocal about the growing cracks in the public education system for decades. We fund our schools through illogical and inadequate measures, creating and reinforcing a system that inherently puts economically disadvantaged students at risk; and students who are overwhelmingly Black and brown or English language learners. We rely too much on testing data to drive funding and policy decisions, knowing these assessments are incredibly stressful for our students. More so, high-stakes testing is a lagging indicator, meaning it is a temperature check confirming the symptoms we already feel.
Research shows high school grades are a stronger predictor of college preparedness than the admission scores most universities require. A 2019 American Education Research Journal study argued that high school grades provide a stronger index of cognitive and self-regulatory competencies. The study was a collaborative effort of more than half a dozen universities across North America. They identified how grades revealed a lot about a student’s ability to be self-aware and more self-directed in their learning than previously believed. In our XQ Learner Outcomes, we call this Learners For Life, research-backed competencies to prepare students for the future. It’s a valuable tool for educators rethinking high school learning outcomes.
When investigating the more granular details of the NAEP results, it’s obvious the sharpest declines would appear in areas where students are more at risk of systemic negligence. Black and Latino students, as well as students who are eligible for free or reduced-price lunch (which is often a proxy for poverty), experienced more significant drops compared to their more affluent peers. The pandemic disproportionately impacted these communities, and students lacked greater access to technology for remote learning.
NAEP’s first post-pandemic assessment confirms what we’ve known for some time now — public education needs more support if it is to work for all students. But the burden isn’t on schools alone.
How to Recover Student Learning Post-Pandemic
I asked Owens about successes in public education we should scale and replicate. “The inclusion of every single learner, no matter language proficiency, income or ability, needs to be seen as one of the brightest examples of American ideals that we’ve ever been able to create through government,” he said, adding that good work is happening. “If you take the top 10 percent of American learners and put them against any other country, our students do as well or better than every other nation,” he noted. “Our main flaw is how do we treat our low-income students?”
To Owens’s point, we already have proven pedagogy. Student-centered and project-based learning leads to stronger academic gains, builds student agency, and supports another XQ Design Principle — meaningful, engaged learning. Access to afterschool programs and small group tutoring outside the school day will also help in pandemic recovery efforts.
Owens added, “If we could shape policies where benefits overwhelmingly support our low-income students, our Black and brown students, we would be the envy of the world.” He pointed to how reading instruction was approached in states such as Mississippi and Florida, shifting to the science of reading instead of mostly phonics-based instruction.
Owens said hiring teachers of color, especially Black and brown educators, can positively impact schools and communities. He also emphasizes the importance of art and music education and the need for unstructured playtime. For high schools, this means increasing access to extra-curricular activities and athletic programs — both of which are proven to help support student learning. Supervised socialization can also help combat growing concerns about time spent on mobile devices and social media, allowing high schoolers to create meaningful friendships with their peers.
At XQ, we believe high schools are a fulcrum point for meaningful change-–because improving the educational experience of high school students can have ripple effects on the expectations for K-8 schools and higher education. Students who took the 2022 NAEP tests are now in high school as 9th graders. But the greater disruption caused by COVID-19 will be felt for the next decade. We must pay special attention to how we respond to these scores and future data sets.
I asked Owens what the general public, parents, and educators should consider regarding data, school funding, and equity measures. “If you have a voice, make sure to use it for all children,” he replied. “If you find yourself pushing for something that will benefit your child, is that something that’s going to help all children, or is that just a way to get your child in the inner circle? Resource and attention allocation are what we’re advocating for,” he added.
Photo at top by Chris Chandler
