Generative AI at BPP: the future of assessment in education
Sarah is a law tutor.
Originally a commercial litigator, she switched to education after discovering a greater passion for teaching than practicing law.
When she started as a tutor one of her core duties was to mark 5000-word essays to assess students’ grasp of material and their critical thinking skills. But things changed significantly with the launch of ChatGPT in 2023.
Students began using the tool for essay-writing, prompting new mandatory training for Sarah on detecting AI-written submissions and how to report academic misconduct. However, the training quickly became obsolete as students adopted more advanced, undetectable tools.
So the decision was made to get rid of essays altogether. The university decided to use exclusively live online interviews to assess students.
New cheating methods emerged for these interviews as well, with students using apps that could listen to questions in real-time, analyse them, and feed the correct answers back into their headphones.
A cycle of deception and countermeasures evolved until eventually Sarah’s role was reduced to such an extent, she now wonders if she is even necessary. She asks the questions, and keeps an eye out for signs of student misconduct, but everything else is overseen by AI.
The entire call is monitored in real time, to detect signs of suspicious behaviour, and grades are awarded based on an automated analysis of the recording.
Hopefully, the world described never becomes reality. Although it might appear to be the direction we’re headed…
Traditionally, the primary form of assessment in all universities is the essay, completed by students in their own time. And here, we have already seen the beginning of Sarah’s story come true.
The most common reaction from universities when ChatGPT first launched was to ban it. Most have since realised this is an impractical strategy, as reliably detecting the use of AI was nearly impossible and has only become more difficult over time.
Higher education is doomed?
I don’t believe so. Sarah’s world will only materialise if universities treat Generative AI as an adversary in a pitched battle.
Them versus us.
Technology versus academic integrity.
We can’t run away from Generative AI, but nor can we bury our heads in the sand. Instead, we need to embrace it and learn as the acceptance and capabilities of Generative AI evolve around us.
At BPP we are in the early stages of exploring the vast capabilities of Generative AI.
Across the university, we are proactively nurturing responsible usage among both staff and students. We have established policies that clearly outline the appropriate and inappropriate uses of tools like ChatGPT. We have training for both staff and students too.
And as the Product & Technology team, we are investigating how Generative AI, as a broad range of technologies, can deliver value for our students and the business.
- Automating mundane tasks
- Content generation
- Enriching existing data
- Language translation and localisation
- Personal assistants, tutors, coaches
- Writing code
But education is the topic closest to my heart. And so, working at a university, I find the ways we use Generative AI to support our core operations particularly interesting. That’s teaching and learning.
I vaguely remember a presentation from an EdTech conference discussing how much of teaching in schools can be automated with the current technology. The presenters claimed that 38% of a teacher’s work is fully automatable, and 53% is partially automatable. Although I’m not completely sure how they derived those values… as somebody who trained as a maths teacher, they sound pretty reasonable. Marking, for example, could definitely be done faster with some AI help!
Let’s talk about how we “partly automated” one of those teaching jobs at BPP…
One of the things that Sarah might do at BPP — as a law tutor — is write practice questions for students preparing for the Solicitors Qualifying Examination (SQE).
Previously, it took our tutors about 1 hour to write a new question that meets the extremely high standard required. And that is in their specialist subject… so 1 hour from a qualified expert, in a single subdomain of law, to create a question that demands critical thinking from the student.
By using prompts refined and automated over several weeks, we can now use Generative AI to produce multiple questions in seconds. The new job for tutors is to review and refine the AI-generated outputs, which only takes 15 minutes per question, freeing up their time for more valuable, human, interactions with students.
That final point is crucial.
Earlier, we identified the automation of mundane tasks as one of the ways Generative AI can benefit businesses.
By automating the aspects of teaching, marking, and content creation that do not need the precious expertise of our experienced tutors, we can increase the time available for more meaningful, human, interactions between tutors and students.
Nobody wants Generative AI to replace human interaction. But we must explore how it can support other, manual, tasks. This way, we can avoid the dystopian future in which Sarah now lives.
