Online proctoring. How it works? Part I.

Hi there!

You are currently reading the Examus blog. It’s worth noting that we develop solutions for online education.

The key mission of the company is to bring online education closer to the world of full-time education in terms of quality and efficiency.

Who can benefit from using it? Everyone who develops remote education systems in educational institutions and companies, as well as HR-managers, employees of IT-departments.

Main topics of this article:

• Structure and operation of the system, and key stages of the examination
• Available proctoring modes

We will try not to go into details; otherwise, it’s going to be a long book, but not an article. At the same time, we will try to give you the most complete picture of the system.

I. Structure and operation of the system, key stages of the examination

The service is integrated into a testing system or so-called LMS (learning management system). It’s worth noting that the Examus online proctoring service is offered as SaaS (Software as a Service), while the entire infrastructure is stored in cloud services. Students does not need to install anything on the computer. They just open the browser, go to the specified link, pass system verification and identity verification, and start taking the exam.

We have to tell you that one can work with the service as through a browser extension, as well as by using a link, but we will talk about this in another article.

Important decision: shall one use a computer or a phone? Our experience shows that only the desktop version gives a reliable result, so we always strongly recommend it to our customers. The mobile version gives the user too many opportunities to exploit the system. However, we understand that in some cases, there is no chance to use a computer, e.g. when field workers are located in a remote plant or factory. We will describe all the features of the mobile version of the system in one of the following articles.

The system requests access to the camera and microphone and checks its functionality, as well as the stability of the Internet connection, the presence of a second screen, and all attempts to simulate or imitate the screen. If the system can pass all the tests, the student has to pass the identification process, i.e. to take a photo and send a photo of themselves with an identity document (it can be a driver’s license, a student ID card, or a credit card).

After that, the student can start the examination. At the same time, the system acquires information from 3 different sources:

• camera
• microphone
• computer screen

After completing the exam, the system will gather and combine information from all the sources into a single combined record.

During the exam, the system analyzes the physical actions of a person and their actions in the environment of the computer. The system is flexible, thus the rules can be adjusted for each examination. In some cases, one can use a calculator, while in others, one needs a paper and a pen.

During the examination, the system automatically detects the following types of violations:

• Changing the direction of the gaze, i.e. when the user is looking somewhere else;
• Lack of the face in the frame, i. e. if the user is not in the field of view of the camera, or the face is not fully visible;
• The presence of other faces in the frame, it is also recommended to remove the portraits of great people from the wall because they can be automatically recognized as violators of the examination rules;
• Substitution of the student, i.e. if the person in the frame is not the same person that was intended to participate in the examination;
• Any human voices in the frame, even if it is the voice of the student (e.g., the student can pronounce questions to another person in the room);
• Change of the windows and tabs on the desktop, i.e. if the user decided to switch from the test window to a search engine or open a third-party application.

In the online proctoring mode, when the entire process is controlled and monitored by a person, the number of violations increases, because the person can notice the violation by indirect signs. The following additional violations are registered:

• The usage of phones, tablets, and other gadgets and devices;
• The usage of TeamViewer, Skype, RemoteAdmin, and other desktop broadcasting software;
• The usage of virtual machines and thin clients to pass an examination by a third party;
• The usage of books, notes, and drafts;
• The usage of calculators (if prohibited);
• All cases of non-verbal communication (winks, motions with hands, nods of the head and so on).

Examus automatically marks all potential rule violations and adds them to the report. Each violation entry contains information about the type of violation, such as moving the eyes away from the screen or changing a window, and a timestamp.

Based on the results of the exam, the system calculates the overall probability of violations during the exam: green zone means low probability, yellow — medium, and red-high.

In other words, we can immediately determine who has to be checked first.

It means that based on the results of the examination, we can do the following actions for each of the students:

• View and listen to recordings from the camera, microphone, and desktop;
• View all potential violations in chronological order;
• By clicking on any violation, one can view the records of what was happening at that particular moment.

II. Proctoring system modes

Automatic mode: in other words, everything is executed without a live operator. The obvious advantage of this mode is that it reduces the overall costs of exams. However, in many situations human participation is simply irreplaceable:

● If the fate of a person depends on the results of an exam, we can’t leave the decision to the machine;

● Even the most precise automation system can’t track certain non-obvious violations;

● At the same time, the automation is very sensitive, and one can’t explain that a stranger in the room is the mother, who accidentally came in and will leave in a matter of seconds;

● There are many cases when one needs to control all the stages of the exam process, e.g. one may need to pass the student by using one of the predefined protocols before giving access to the examination;

● The operator of the proctoring system can make sure that the student follows the procedure for a particular exam. In some cases, the student must show that their palms are not full of hints and answers, while after another exam, they must tear the draft into small pieces;

Therefore in addition to automatic proctoring, there are 2 modes with human participation:

1. Asynchronous mode: it means that an operator will review the recording after the examination;
2. Synchronous mode: it involves remote online monitoring and control of the examination process by an operator of the proctoring system;
3. Identification mode: in this case, the operator only confirms the identity of the student before entering the exam;

Each mode has its interface and methodology for working with operators, which we will discuss in detail in the following articles.

Resume

So we hope now you know how the proctoring system works in general.

In next part we’ll delight by what technologies it supported.

Share the most interesting topics in the comments.

Don’t forget to subscribe to our blog!

Sincerely yours, Examus Team