Edtech & the 2022 PISA rankings
In December 2023, the OECD released the data of the 2022 PISA (Programme for International Student Assessment) rankings, that are normally released every three years, but have been pushed back by a year due to the pandemic and its profound effects. The findings are highly compelling and worrying. They also carry valuable lessons for both edtech operators and policy makers.
The pandemic-induced disruption alone cannot account for the unprecedented decline in performance
It’s widely acknowledged that the structural challenges the global education landscape faces extend far beyond the sole ramifications of the Covid-19 pandemic-induced disruptions. From resource constraints, unequal access to high-quality learning content, to outdated teaching methods, under-digitization and curriculum disparities, education systems around the world face significant hurdles in their quest to provide quality education to all. These challenges are now well-documented by scientific research. The 2022 PISA results provide further quantitative evidence of these issues in OECD countries.
Overall, the state of global education is worrying. The global trend the report highlights is that average student performance in reading, mathematics and science is decreasing:
“Some 25% of 15-year-olds in OECD member countries — representing 16 million children — are estimated to be low performers in maths, reading and science (…) they can struggle to do tasks such as use basic algorithms or interpret simple texts”
This drop in performance is unprecedented — students on average scored 10 points less in reading and 15 in maths vs. 2018. The graph below suggests nonetheless that mean student performance has been on a declining path long before the pandemic hit:
This does not mean of course that the pandemic did not negatively affect education systems across the world but it unequivocally highlights that “educational trajectories were negative well before the pandemic hit [indicating] that long term issues in education systems are also to blame for the drop in performance”.
It also provides interesting insights for policy makers when it comes to the benefits of edtech tools, when used appropriately, as pointed out in the infographic below:
What the 2022 PISA results tell us in numbers
An overview of the data in mathematics, reading and science
In most OECD countries, particularly across Europe, the prevailing trend in the fundamental subjects under examination (mathematics, reading, and science) is exhibiting a negative trajectory. The overall performance in PISA 2022 fell markedly below the average performance observed since the initiation of PISA assessments. Worse, some countries have experienced a “hump-shaped performance”, improving at first before engaging in a downward slope in recent years (Spain and Poland in Mathematics for instance, Germany and France in reading, Italy and Estonia in science)
In general, with the exception of Italy and Portugal, no European country has experienced an improvement in average mathematics performance. Below are the results in reading and in science.
Overall, the performance of European countries is negative. No country is on an upward trajectory across all subjects.
What explains the discrepancies in results between surveyed country?
Certainly, Covid-19 has contributed to the worsening of students’ average performance in all disciplines covered by the PISA assessment. On average, OECD students missed 98 days of class during the lockdowns. The short-term effects, both in terms of academic performance and mental health, have been extensively documented by scientific research (see here for instance). The medium and long-term effects, on the other hand, are just beginning to be felt and will be more directly observable in the years to come.
The PISA student confirms that « systems that spared more students from longer closures scored higher in mathematics and reported a greater sense of belonging at school.” (page 23,n Vol 2)
However, upon closer examination, the differences in school closure policies among countries (English students missed 88 days of classes, compared to 65 in Italy or 85 in Germany, and 67 in Spain), and the lack of exact correspondence with academic performance, seem to indicate that school closures alone are not enough to explain the drop in performance. France, in this regard, provides an interesting paradox — students’ performance between 2018 and 2022 declined more than in other OECD countries where the number of missed school days by students was higher.
Therefore, the reasons must be sought elsewhere and are of a structural nature. The inherent characteristics of educational systems plays a role in educational performance. Covid-19, which acted as a “stress test for resilience in education” has hence highlighted the “components for resilience”. They are numerous and varied— ranging from educational policies to the involvement of the teaching staff, and the mobilization of edtech tools.
PISA components of resilience :
Overall, there are thus various “components of resilience” and they can be categorized into two main dimensions: temporary policy choices and structural orientations.
Temporary policy choices refer to specific decisions made in response to certain circumstances during Covid19 — for instance, the duration of school closures, the attention given to students’ well-being during lockdowns, and efforts to maintain social connection and a sense of belonging despite school closures.
On the other hand, structural orientations are broader and relate to the overall design and approach of the educational system. One important aspect is the delivery of distance and online learning. The way in which these modes of learning are implemented and integrated into the educational system is crucial for building resilience. This includes the system’s capacity to prepare students and teachers to effectively utilize these tools and to develop “self-directed learning skills” that enable independent and autonomous learning. The PISA report indicates significant variance among OECD countries in this regard. In Europe, Estonia, Finland, Italy, Sweden, and Switzerland have a “solid foundation for remote and more autonomous learning.” The following table demonstrates this correlation:
In general, the difficulties in deploying remote learning are linked, on the one hand, to technical challenges (such as access to internet connection and the inability to find a suitable environment for studying) and, on the other hand, to pedagogical difficulties that could be described as instructional in nature (comprehension of assigned tasks, inability to find someone to provide assistance, etc). This second set of challenges suggests that one of the issues with remote learning is not so much related to the tools used but rather to how they are implemented and integrated into the pedagogical and learning framework. Once again, it becomes quite evident that this is not solely attributable to the Covid-19 situation:
Education systems in which fewer students reported problems with remote learning also had more positive 2018-2022 performance trends […] when analysed in relation to longer-term trends (i.e. “adjusted short term trends”), even though no significant relationship was observed to the 2018–2022 performance trends, when longer-term trends were not considered (i.e. “unadjusted short-term trends”). (page 81, Vol 2)
How edtech can help
The solutions to the challenges outlined in the PISA report will not solely come from remote learning. However, it is evident that edtech has a fundamental role to play in addressing these challenges. This role encompasses various aspects and for several reasons. According to the PISA report, it emerges that education expenditures only affect student performance up to a threshold of $75,000 per student. Beyond this threshold, there is no longer a correlation between extra investment and student performance. Therefore, the answer lies in the allocation of resources.
In this regard, by enabling pedagogy to scale, for example, through the use of AI like EvidenceB, edtech could optimize the allocation of resources. This becomes even more crucial when it comes to underperforming students who require adequate resources and support.
The PISA report highlights the value of greater personalization, the use of analytics, and the adaptive and interactive nature of learning experiences. These elements play a significant role in enhancing the effectiveness of education.
Personalization allows for tailoring learning experiences to individual students’ needs, strengths, and interests, promoting engagement and motivation. Analytics provide insights into students’ progress, strengths, and areas for improvement, enabling educators to make data-informed decisions and provide targeted support.
The adaptive and interactive nature of learning experiences allows for adjusting the content, pace, and level of difficulty to meet individual students’ learning requirements. This helps students stay challenged without feeling overwhelmed or bored, fostering a more effective learning process.
Game-based learning, mentioned in the PISA report, has gained a lot of traction in recent years — companies like PowerZ or complori being relevant examples of this.
Edtech can thus help personalize and target interventions, ensuring that these students receive the necessary assistance and resources to improve their performance, bridging the education equity gap.
Similarly, teachers, who play an essential role in learning, may not have the necessary time to assist students. The analysis of PISA highlights a paradox: the number of teachers per student has remained stable or increased in OECD countries, yet there is a strong sense that there are more teacher shortages than ever before. Tools like Nolej, an authoring tool that harnesses the power of AI to assist teachers in developing engaging educational content. By leveraging AI, teachers can streamline content creation processes, saving valuable time in their instructional preparation. This time-saving aspect allows teachers to allocate more of their precious time and attention to supporting and guiding their students. As the report points out, “the effectiveness of education can never exceed the quality of teaching and teacher support.”
Overall, and as always, it is the way edtech tools are used that matters — their impact varies depending on the pedagogical frameworks they support, their integration into the wider instructional setup, and the public policy choices made regarding their deployment.
