Does data-driven learning improve equity?
Student data matters. But it’s not all that matters.
An underlying assumption driving the use of student data is that it introduces objectivity, increases efficiency, and lessens discrimination in an evaluative process that has traditionally been heavily influenced by subjectivity. And yet, assumptions and expectations are part of the process of data collection, analysis, and interpretation.
Data has the potential to enhance educational opportunities for all students, but data alone does not guarantee equity.
The use of educational data is not the first method that has been proposed to promote equality in education, nor is it new. Using data from standardized tests to inform educational decisions has a long history. The original purpose of standardized testing was to develop an objective national baseline measure of student performance that could potentially improve school efficiency. Considering the value American culture places on meritocracy — the idea that each person’s success is attributable to their own objective performance — turning to standardization to create fairness in education is not surprising. However, since the widespread implementation of standardized testing in schools, persistent achievement gaps in underserved communities have been a problem.
Student data can and has served as an equalizer, but it also has the potential to perpetuate discriminatory practices. In order to leverage student data to move toward equity in education, researchers, parents, and educators must be aware of the ways in which data serves to equalize as well as disenfranchise. Common discourse surrounding data as an equalizer can fall along a spectrum of “yes, it’s the fix” or “this will never work.” Reality is more complicated than that. Real life examples of the use of student data show that:
- Data can raise awareness of disparity or be used to justify discrimination.
- Data can be used for inclusion or exclusion.
- Data can be used to inform transformational educational intervention or to game the system.
- Interpretation of data can lead educators to provide extra support for students in need or support biases.
Raising Awareness vs. Justifying Discrimination
Educational data can be used to call attention to inequality, but it can also be used to justify discrimination. One example dates back to 1956 when Washington D.C. schools instituted ability grouping that resulted in blatant segregation in schools, with white students mostly placed in higher-performing tracks and black students assigned to mostly lower tracks (Arthur R. Jensen tells the story in chapter 2 of Bias in Mental Testing). The ability grouping was based primarily on achievement and aptitude test scores. The apparent segregation led to the Hobson v. Hansen 1967 court case — the first case in which criticism of aptitude tests played a central role.
Yet datasets collected over time or across groups can also uncover persistent discriminatory patterns beyond isolated incidents. Recent analyses have shown that there are disturbing racial disparities in classroom disciplinary practices: youth of color face harsher discipline than white students; they’re referred to law enforcement more often; and they’re more likely to be taught by lower-paid and less experienced teachers. The Pew Research Center used data analytics to call attention to ongoing segregation of school systems in the US. Through the analysis of large datasets, disparities can be quantified and lead to awareness-raising efforts and challenge practice.
Providing Extra Supports vs. Devoting Fewer Resources
Student data takes on completely different meanings depending on the educator’s interpretation, and often teachers are not taught how to interpret and act upon student data. In some cases, data may result in educators paying more focused attention to struggling students while in others the educator may decide to devote fewer resources to them. Educators may form assumptions about student capability, effectively reducing kids to a category. In a study of Chicago high schools, test scores led teachers to differentiate between “deserving” and “undeserving” students. Teachers attributed “undeserving” students’ poor performance to a lack of motivation, resulting in limited learning opportunities for the students.
Schools can also target early intervention operations through the use of predictive analytics tools to identify students who are at risk of dropping out. One example comes from the Tacoma school district in Washington, where five high schools were identified as “dropout factories.” In 2010, only 55% of the district’s high school students earned their diplomas on time. To raise graduation rates, the district worked with Microsoft to analyze student data to predict risk of dropout. Reportedly, the model was nearly 90% accurate in identifying students for intervention. As a result, graduation rates increased from 55% in 2010 to 78% by 2014. The biggest gains in graduation rates were seen in low income and minority students.
Inclusion vs. Exclusion
The promise of data in education is that it can identify and amend exclusionary practices. An FTC report shows the benefits of longitudinal data analysis. A review of a school district in North Carolina found that 20% of students who had been determined ineligible for eighth grade math by their teachers were eligible for advancement based on scoring. When advanced based on scoring rather than teacher recommendation, 97% of these students easily completed eighth grade algebra. Analysis of student data provided a much needed course correction for advancing students in these schools and also flagged biased, prejudicial practices.
Yet data can also be used to justify and support biases. In 2015, This American Life detailed the story of a school district in Normandy, Missouri. The district was mostly black, poor, and suffered low test scores and minimal graduation rates. Students from Normandy had the option to attend a higher performing and mostly white school in their district. But parents at the higher performing school used crime statistics from the Normandy school district area to rationalize blocking the students from entering their school. Others expressed concern that students who received low test scores in their home district would bring down performance averages at the higher performing school. Parents essentially used data to justify the exclusion of these students.
Transforming Educational Practices vs. Gaming the System
Data can drive decisions that improve educational practice or attempt to obfuscate poor performance. In a study of school districts in Chicago and Texas, researchers found that, for schools placed on academic probation, data reporting was used to avoid penalties without improving educational practice. Yet in schools with histories of high student achievement, interpretation of test data led to school-wide instructional improvement.
It’s not just about having the data but about who has the resources to use it.
In this case, the schools that already had more resources improved, widening the gap between the better performing schools and their lower performing counterparts.
What Does This All Mean?
The use of student data can greatly reduce bias in education and help students reach their potential. But data itself is dependent on the measures chosen. As the history of standardized testing shows, these measures can unintentionally privilege certain groups over others. Entirely removing human bias is not a reasonable expectation, but data may be able to reduce it. Does the current use of student data shift us closer to equity or does it justify the status quo?
The reliance on data from standardized measures is in the aim of establishing a meritocracy, but turning to meritocratic ideals to attempt to solve the problem of inequity is naïve. An entirely merit-based system would deny external socio-economic factors, turning instead to a focus on performance. If someone fails to succeed in a meritocracy, it is assumed to be a result of personal inefficiency.
The assumption of “objective data” frees people from acknowledging structural inequality.
Failure to acknowledge structural inequality in relation to educational data neglects issues impacting disadvantaged students. How do we leverage it in a way that doesn’t allow for the cloaked continuation of discrimination? Quantitative data seems objective, because numbers are assumed to be factual representations of reality. As the above examples show, the road to achieving equality is complicated.
Points: “Does data-driven learning improve equity?” That depends, says Mikaela Pitcan in this Points original. Starting assumptions, actual data use practices, interpretation, context context context — all complicate the story around education data and must be kept in mind if equity is our objective. For more on data, tech, and learning, follow Data & Society’s Enabling Connected Learning initiative on Medium.— Ed.
Mikaela Pitcan is a doctoral student in Counseling Psychology at Fordham University and a research analyst in the Enabling Connected Learning initiative at Data & Society.
