To achieve top global education goal, don’t stumble on measurement
In the impassioned debates surrounding standardized testing and how best to measure student learning, a key fact is often forgotten: assessments are not an end in themselves, but the means to an end. They provide a better understanding of what children are learning (or not) — information that can be used to improve curricula, instruction and ultimately student learning.
There are many different paths to this information. Competent assessments, no matter their design, shed critical light on how to improve results — the key is in actually using the data. Education practitioners must be mindful of this truth as they pursue the most ambitious education improvement initiative in history.
Last month, at the launch of a new Global Alliance to Monitor Learning hosted by the UNESCO Institute for Statistics (UIS) and the World Bank, donors, researchers, practitioners and technocrats met to review options for reporting on Sustainable Development Goal (SDG) 4.1, which pledges that by 2030, countries will “ensure that all girls and boys complete free, equitable and quality primary and secondary education leading to relevant and effective learning outcomes.” To measure progress against this goal, countries will report on the percentage of children/young people (i) in grade 2/3, (ii) at the end of primary and (iii) and the end of lower secondary achieving at least minimum proficiency in reading and mathematics.
In the midst of a robust movement in the US to “opt out” of standardized testing, it is noteworthy that many other countries are asking for more. But throughout the SDG process, stakeholders were adamant that learning, not just access to schooling, would be central to the new education goal. So how can UIS, which has the mandate to monitor progress toward achieving this goal, knit together a complex patchwork of international, regional and national assessments to report on learning? The crux of the issue lies in what is meant by “comparable.” In a recent blog, the head of UIS lamented that “each assessment has its own framework and methodology, making it impossible to compare the results.” But surely we can sort out some valid form of comparison (note that contrary to the cliché, apples and oranges are both round, can be peeled, have seeds and grow on trees). Knowing that technocrats have a tendency to make things more technical than perhaps need be, this is a case where a simple approach that builds on existing efforts is the best solution.
According to UIS data, most countries have conducted learning assessments at one or more of the required grade levels. Using those results, countries could define what is meant by “minimum proficiency in reading and mathematics” and report on the percentage of children that meet or exceed that standard. In Washington DC, where my children attend public school, 25% of third graders score proficient or better in reading; in math it is 30%. These data can readily be compared to the percentage of children meeting local standards in other states and countries. Might education officials in DC be defining proficiency a bit differently than ministry officials in Denmark or Djibouti? Probably. But does it matter? As long as parents, practitioners and policymakers are using the data to push for better learning outcomes, it should not matter if each country measures learning in exactly the same way. While perfectly equivalent measurement might make life easier for bureaucrats in international agencies, insisting on a single assessment approach is likely to generate (even more) resistance from teachers and communities. In our search for data perfection we sometimes lose sight of what really matters: if and how the results are used to improve learning.
Global reporting should rely on the best available student assessment data countries can offer, whether national, regional or international. That doesn’t mean that countries can ask kids to self-report if they know how to read. Assessment should adhere to minimum standards of technical quality and countries should use results to set reasonable (not too high, not too low) performance expectations. Global agencies like UIS, USAID and others should continue to help countries improve their methods. The World Bank provides ratings on the quality of national student assessment systems, this framework could be expanded to additional countries. And UIS is collaborating with the Australian Council for Education Research to work out a system that can put all of these assessment results onto a single scale.
The education sector should also reflect on lessons from health. Calculation of the under-five mortality rate relies on country-based systems for reporting from a variety of instruments. The World Health Organization provides guidance on how to measure the number of under-five deaths as a proportion of live births using all available national data, drawing from sources as diverse as hospital records or household surveys.
While there has been debate in the health community that some countries may be cheating (excluding babies of early gestational age or low birth weight), the indicator has had a powerful effect on maternal and child health advocacy. During the 1980s campaign for child survival, UNICEF and its allies relied heavily on the indicator to issue a call for action to ensure higher numbers of children would live past their fifth birthday. It worked.
Education is at a similar point to where health was in the 1980s. We need to ensure that children don’t just have access to school, but that they thrive and learn throughout schooling (and beyond). And we need to use the best data we can get our hands on to drive that message forward. In the spirit of the broad consultative process that characterized the development of the SDGs, UIS should encourage countries to measure and report on what they mean by proficient in reading and math. UIS can and should continue to play a role in both data aggregation and reporting and set global standards for what qualifies as quality measurement. But it should not let the perfect be the enemy of the good, or insist on a single approach. To do so would risk countries rejecting any global reporting on learning. We want countries to measure learning in a way that is most relevant and useful to them, because then there is a greater chance that they will use that data to improve learning in their classrooms.
