AI-enabled Assessment & Grading Landscape
By Avner Point & Alexandre Glaser
The debate about teachers’ working conditions is not new. For several years, if not decades, numerous research institutes, surveys, and NGOs have been issuing warnings about the deterioration of these working conditions. The general consensus is that teachers are overworked, underpaid in most countries and under-equipped to do their job. In the UK for instance, a report showed almost “three-quarters of teachers said their workload was unacceptable” (Source). Almost 25% of surveyed teachers declared working 60h/w or more. In the US, according to this study, teachers worked 53 hours per work, for an average salary that only 34% of them said was acceptable (vs. 61% of working adults). In the UK, this unsustainable situation is leading to 81% teacher considering leaving the profession.
A new path to better allocating teachers’ time
One solution is thus to raise the salaries of teachers. This increase should nevertheless be backed up by a policy to improve working conditions.
In particular, this means rethinking and optimising (i.e. reducing) the allocation of teachers’ time to tasks that they regard as repetitive on the one hand, and of little value in the knowledge transmission on the other. According to a McKinsey study teacher spend only 49% of their time in direct interaction with students:
Against a backdrop of a general decline in student level, as highlighted by the results of PISA 2023, and difficulties in recruiting teachers, this optimisation of time allocation seems all the more critical.
On the other side of the fence, students, and in particular the younger generations, experience a double frustration with the assessment process. Firstly, many believe that the rating and assessment process is subject to numerous biases. Scientific research has clearly established the frequency of these biases, as well as their nature — they can be name-based, background-based, or linked to the previous performance of the student in question, among other things [See here, here and here for instance]. At the same time, a number of assessment techniques and procedures have been put in place other the years to reduce these biases: anonymous exam papers and double marking, for example. One of the most effective methods, however, is to reduce the opportunities for biased decision making in the grading process.
Secondly, the grading process is time consuming in itself. This leads to visible inefficiencies, with students often waiting a very long time for their marks and feedback. This would not in itself be a problem if the assessment did not allow certain mistakes or limits in the student’s understanding of a topic or concept to be identified and corrected early enough to prevent students from repeating them. But one of the advantages of grading and assessment is precisely that it enables to assess a student’s level at a given moment, so that they can improve.
Not a new theme but new tools
Tech-enabled evaluation and grading is not a new theme in itself. The edtech sector has often seen this segment as a fertile ground for innovation: the technological and adoption barriers are relatively low, and the gains for users are immediate. Numerous tools have enabled teachers to assess their students using multiple choice questions or simple assessment formats. Since Kahoot, among others, gamified and collaborative assessment has become widespread.
Some subjects lent themselves to this more than others — those where answers could be assessed using a true/false system, for example. But the widespread access to generative AI has opened the way to new, more complex automated assessment practices that are better able to reflect the different ways in which students are assessed (open-ended questions, inverted classroom, practical exercises, etc.). In this respect, the assessment value chain is experiencing the emergence of new players while established players are launching new modules leveraging generative AI to reduce the workload of teachers and improve the assessment process as a whole.
The Assessment Value Chain & Landscape
1. Exam & Exercises Creation
The process of exam creation encompasses all the steps involved in crafting an assessment document. This category of products’ features range from assisting teachers in formalizing documents, quizzes by editing typos, questions, and scoring to generating AI-based documents that can autonomously create questions tailored to the course content. AI-enabled tools allow instructors to engage students more effectively by personalizing exams to suit individual learning styles and preferences. These advancements also facilitate increased collaboration among students and foster inclusivity through gamification features, resulting in a more dynamic and engaging learning experience. For example, Nolej allows teachers to introduce a variety of different exercises format (find & drag the word, open ended quizzes, flashcard exercises, question sets, crosswords) just like an enhanced authoring tool.
2.Exam Delivery & Proctoring
Online proctoring, which became a major theme during COVID-19, has emerged as a critical theme in Edtech for maintaining exam integrity in remote learning environments.
This sub-market is dominated by a handful of key actors (inc. Measure Learning — ex. ProctorU), and has been consolidating over the years. Key drivers of success remain the efficacy and the reliability of the tech rolled out. In recent years we saw these actors leverage advanced behavioral tracking, eye monitoring, and AI-enabled detection of writing styles, along with browser integrity monitoring. Innovation also extends to the pedagogical formats of proctoring (live, pre-recorded, and hybrid approaches) to cater to diverse educational needs and preferences.
3. Plagiarism Checker
Just as the proctoring sub-market is not new in itself, the plagiarism sub-market has existed for a number of years. With the emergence of generative AI and sophisticated writing assistants, the challenges have multiplied. The primary purpose of these tools is not to punish students; instead, their aim is to promote genuine understanding and knowledge. Nevertheless, research from Stanford indicates that this technological advancement has not led to an increase in cheating in high schools. As Victor Lee from Stanford GSE writes: “But I think it’s important to point out that, in Challenge Success’ most recent survey, students were also asked if and how they felt an AI chatbot like ChatGPT should be allowed for school-related tasks. Many said they thought it should be acceptable for “starter” purposes, like explaining a new concept or generating ideas for a paper. But the vast majority said that using a chatbot to write an entire paper should never be allowed. So this idea that students who’ve never cheated before are going to suddenly run amok and have AI write all of their papers appears unfounded.”
Two types of solutions dominate the market — plagiarism checkers for academic institutions, the leader being Turnitin, and reformulation and paper-writing support tools for students.
4. Grading, Assessment & Feedback
AI in edtech as written extensively (see here) serves as an assistant, facilitating tasks for both educators and students rather than replacing them outright. One prominent application is thus in grading and assessing papers, offering significant time-saving benefits. McKinsey reports that grading and feedback consume approximately 6.5 hours out of the available 50 hours per week for educators. Utilizing AI technologies can potentially reclaim three hours of this valuable time for other activities, including personalized feedback and student mentoring on specific tasks.
With the advent of GenAI we’re seeing a wide range of sophisticated point solutions for grading and assessment that allow for enhanced automated personalized feedback and to push the right exercises to the right student. As such, they act as a teacher multiplier of sorts, increasing the teacher-to-student ratio, while alleviating their workload. Simultaneously it allows to reduce the grading time and thus help students get access to feedback much more rapidly.
Automated grading is still in its infancy, and we expect to see many companies scaling up in this sub-segment of edtech. If you’re building in this space, drop us a line!
