COVID-19: Cheating On Remote Tests

With AP tests and the LSAT now moving online, take a look at the technologies tasked with preventing rampant cheating.

The Law School Admissions Council (LSAC) finally announced the inevitable on Wednesday: the April LSAT is cancelled. What was less certain was what they would do (if anything) to replace the test. Well, like the AP tests, the LSAT is going online in May.

Immediately I started seeing complaints from people on Reddit and Twitter who are worried about people cheating on the digital test and gaining an advantage over them. To be fair, both sides are in a tough spot: students are right to be concerned about cheating, but what option does LSAC have? Cancelling the test outright would mean that both March and April registrants would have to wait until at least June to take the test.

In this post I want to focus on ProctorU, the digital test proctoring and security solution that the LSAT will use for the first time, and that many other tests and schools already use today. Even after the COVID-19 pandemic subsides, I expect technologies like ProctorU to be become a bigger part of students’ lives as more testing goes online.

ProctorU employs human proctors that monitor students throughout the exam.

The live proctors first assist students with launching the exam and verifying their identifies using multi-factor verification. Available verification methods include public records challenge questions (where you verify previous addresses, lenders, etc.), keystroke biometrics, ID capture, and facial analysis.

The proctors then ask students to pick up their computers and pan their webcams around the room so that they can verify there are no prohibited items in the room. According to ProctorU’s founder, “…an average 80% of the test takers have to be stopped before the exam begins to remove something from the testing area, such as a cell phone.”

The proctors keep monitoring throughout the test, and intervene in the event of any suspicious behavior. “Proctors will stop 7% of test takers once during the test for engaging in suspicious behavior, 2% of test takers will be stopped twice during a test, and 1% of the test takers keep pushing the limits and have to be reported for cheating.”

So what exactly counts as a suspicious behavior? ProctorU’s website lists these examples:

• Looking off screen
• Accessing prohibited materials
• Speaking out loud
• Allowing other people into the room

So what happens if a proctor notices one of these behaviors?

“During a Live+ exam, our proctors maintain the ability to use the mouse of a test-taker’s computer and can communicate with them through the webcam. In the event that suspicious behavior is suspected, our Intervention Specialists will be invited to the session. These specialists’ sole responsibility is to identify and prevent integrity violations, as defined by your organization.”

Sounds pretty intense. But obviously human proctors can’t catch everything. Since there isn’t a proctor assigned to every single test-taker, it’s possible for things to slip through the cracks. Which is why Proctor U also uses…

Artificial Intelligence continuously monitors test takers and automatically flags suspicious behavior

Yep, you’re always being watched! ProctorU’s technology can recognize and alert proctors to eye movements, low audible voices, slight lighting variations, and other “behavioral cues.”

What’s more, students give complete permission to ProctorU to view everything on their screen. Opening new applications or browser tabs gets flagged, so there’s no way to transmit questions through the device students use to take the test.

To me, the biggest gain to exam security by adding in artificial intelligence is automatically flagging students who are constantly looking at the same spot off screen. This effectively stops students from using a cell phone to relay answers to other test-takers.

Take this story from ProctorU:

“A few years back, a proctor caught a student cheating who wanted to sit on the floor. But when his eyes kept looking at the same spot on the floor, the system’s artificial intelligence alerted the proctor that the student’s eye movements were suspicious. As it turns out, the student had notes tucked under his rug, and the proctor caught him cheating. Now, test takers aren’t allowed to sit on the floor.”

Every test’s system for detecting cheating will be different

For example, the College Board has already announced that AP tests will be open book and open note. So it appears that they will either employ a modified version of ProctorU’s live monitoring system, or not use it at all. Students will have to verify their identity, but it appears that more security measures will kick in after the test, to detect plagiarism:

“We’re using a range of digital security tools and techniques, including plagiarism detection software and post-administration analytics, to protect the integrity of the exams. In addition, each student’s AP teacher will receive copies of the work the student submits to us, enabling teachers to spot inconsistencies with students’ known work.”

No monitoring system is perfect. There will always be cheaters who find a way to circumvent the system.

Before a new method of cheating is discovered, test takers will be able to take advantage of that gap in the system. Even after the rug trick was discovered, people have tried some crazy things to cheat on ProctorU tests, including having a dog enter the room with post-it notes stuck to it, and even a student who used a drone flying in the room to take pictures of the screen! Now both drones and pets are explicitly prohibited.

But it’s not hard to imagine makeshift systems that people could use to thwart ProctorU’s monitoring capabilities. For example, a system of 5 buttons on the ground under a desk could be used to transmit A, B, C, D, and E answers without ever looking off screen or making any noticeable movements for the artificial intelligence to pick up on.

Students just need to be comfortable with the reality that cheating is possible. That goes for remote and in-person tests.

Is it fair? No. Is it better than completely cancelling tests? I think so.