The Myth of the Cheat Proof Digital Exam
Why it is impossible to lock someone out of their own computer
The age-old tradition of pen-and-paper exams has barely changed at all over the last century, and it carries with it a lot of burden. They are costly and environmentally unfriendly to print. Students complain of hand cramps and teachers grumble over unreadable handwriting. And what to do when one of the students’ exams is misplaced and then trashed by the janitor?
Of course, there is a reason exams have not yet experienced the digital revolution that most other parts of our society have. If you let students write exams on their own computers, the technology not only streamlines the process, but also inadvertently gifts the test takers with thousands of new and imaginative ways to cheat the system — smuggling hand-written notes will seem like the stone age when compared to Wikipedia. The alternative, schools supplying a trusted test device to each student in the class, is an economical and administrative nightmare.
This has not stopped a handful of young startups from trying to tackle this problem. The most successful one I have heard of, the Swedish company DigiExam, promises the best of both worlds: Students can bring their own devices to the exam hall, and their system will ensure that they can not use the computers to cheat while writing their answers.
The company has seen a fair bit of success. The application is used at more than 600 schools in over 40 countries, including more prestigious ones such as the Stockholm School of Economics and Columbia University.
Unsurprisingly, the cheat proof aspect plays an important role in their marketing: On the website they boldly label themselves as “Easy to use — Cheat proof — Reliable”. A slogan that, unfortunately, falls flat as soon as you realise it would take a competent student no more than 15 minutes to circumvent every single safeguard they have put in place.
To be fair, it must be said that DigiExam has made admirable effort to prevent abuse. When the exam has started the student can do nothing but answer the questions — no switching to Wikipedia to check some quick facts. Full kiosk mode is enforced, and with it the menu bar, desktop switching, and all other functions not strictly necessary for completing the exam are disabled. Scheduling a script to pause the process at system level after the exam has started will just leave you with an unresponsive screen. You might try to bypass all this by opening the app in a virtual machine, but you will find that DigiExam easily detects the VM and shuts itself down.
As said, it is an admirable effort, but ultimately futile. Because the challenge they are trying to solve is — by definition — impossible.
When somebody owns a computer, that in all likelihood means they have root access to it. And when they have root access, they are capable of changing the behaviour of any program that runs on it in any way they’d like.
How To Disable Cheat Protection in Any Digital Exam
An application is essentially just a bunch of machine instructions: a collection of ones and zeroes that tells the computer which commands to execute. If you want to change the behaviour of an application you have been given, the most reliable and universal way of doing that is to directly edit these machine instructions — changing the ones and zeroes–to do something else.
To do this you will need a disassembler (for OS X, I recommend Hopper) and/or a hex editor (such as Hex Fiend).
Now, I will not show you how to break DigiExam specifically, as I would n0t want to make it entirely too easy for an enterprising student to use this article in order to gain an unfair advantage. But I will tell you the general process of how to disable certain parts of an application, a method which can be applied to DigiExam as well as any other digital exam software.
The first step is to disassemble the application (convert it from machine code to a more readable format) using a disassembler such as Hopper. Then follows some detective work were we search the disassembled application for the method responsible for enabling cheat protection, either by following its flow of execution or by searching directly among the names of its methods and variables.
When we have found the part of the program that we want to disable, we neuter it by modifying or removing some of the machine instructions so that it no longer performs its intended function. We can either do this in the disassembler and then reassemble the program, or we can edit the binary of the application directly using a hex editor.
Generally, only small adjustments are necessary. Disabling an entire section of code often requires nothing more than setting the value of global constant to 0 instead of 1, or changing a jump if equal (je) instruction to its oppositejump if not equal (jne).
In fact, to disable every kind of cheat protection in DigiExam the student only needs to modify two machine instructions at two specific places. This takes no more than 15 minutes — 10 minutes to find the relevant sections and 5 minutes to make the changes.
I can not understate the inevitability of this exploit, as there is nothing DigiExam can do to prevent this. Any safeguards they attempt to construct, no matter how complex, can (and will) ultimately be dismantled by someone using this technique.
The only sliver of protection available is to employ what is known as obfuscation and anti-tampering techniques. These increase the time investment required by making it harder (but not impossible) for malicious users to explore and understand the codebase.
Recently, advanced (and ghoulishly expensive) obfuscation software has miraculously increased the time until cracked versions of AAA video games hit the internet from a few days to to a few weeks or months. This works wonders in the video game industry, where the majority of units are sold immediately after release, but is only a small comfort for an exam software that is meant to be used indefinitely.
The Full Extent of the Problem
So digital exams on personal computers can never be trusted, but why should we care? It is a fair question. After all, regular exams can be cheated as well. And if the student needs access special technical knowledge to do so, that does not seem very problematic.
But in reality, it will require neither knowledge nor effort. The application needs only be cracked once by a single individual, such as me, who can then upload and share the corrupted version with either their friends or with all 600 schools.
As such, a widespread adoption of digital exams could enable a black market of cheating software. This is especially true for DigiExam, whose exam interface is just a basic webview. With a compromised version of the application you can easily write or download simple JavaScript extensions to help you in various clever ways. Think Chrome App Store, but for cheating.
And while cheating on a regular exam is confined to concealed notes or peeking at your neighbours desk, in the digital world only your imagination sets the limit. Automatic spelling and grammar correction? Piece of cake. An extension that grabs and copies the answers from a friend writing the same exam? Sure. It could even paste the answers letter for letter in sync to random tapping on the keyboard, to make it appear as if typed out by hand. If you can dream it, you can do it.
In spite of this, the Swedish National Agency for Education has decided DigiExam fulfills its criteria for a secure digital examination. With this official seal of approval the application was used by more than twenty thousand Swedish students when writing the national subject exams this spring.
So maybe next year, upping your grade on the national exam could be as easy as finding your favourite among a growing batch of cheating software. A digital revolution for sure — but is it in the right direction?
Addendum: There has been a misconception among some readers that DigiExam is meant to be used by the students at home. This is not the case — the software is meant to be used at school with exam overseers, but using the student’s own computer or device.
A note about responsible disclosure
Usually when I discover a security flaw in a piece of software or on a website I contact the authors in private and give them time to fix the issue before going public, a process known as responsible disclosure.
This, however, is not a fixable bug. This in an inherent and unfixable flaw with the entire concept of digital exam software. Therefore, while I have contacted DigiExam ahead of time and notified them about this article, I felt I was unable to offer them a timeframe in which to “fix” it. Furthermore, responsible disclosure also entails disclosing vulnerabilities to the public as early as possible to allow the users to make informed decisions as to the safety and security of the tools they employ.
DigiExam has been given the opportunity to read this article and confirm it contains no inaccuracies prior to publication.
