Ideal exam format: 12 key features

School and college assessment fail at being authentic and relevant. Let’s change this.

Photo by Ben Mullins on Unsplash

Teachers, what is your idea of an ideal assessment?

Pedagogical theory points at precision, reliability and validity (remember the arrows and the target analogy?). But how do we operationalize these theoretical abstractions?

Here is my take on ideal assessment in school, college, and university settings.
I am calling it the SAIL format.

SAIL combines the best practices in assessment into a single exam format.

A SAIL exam must meet at least 6 of the following criteria (first 2 are required):

1 A variety of question formats

A well designed assessment should use a variety of question types.
These questions types can include:

• Fill in the blanks
• Single best answer, multiple-choice questions (MCQs)
• Multiselect (‘select all that apply’) questions
• Jumbled sequence (‘Reorder into the appropriate sequence’)
• Short-answer questions
• Assessment of probabilistic thinking: assign verbal probability terms (such as most likely, probable, possible, unlikely, very unlikely) to answer options.

But no true/false, except, never, all/none of the above type questions.

2 Resources provided to students during the exam

Detailed formulas, constants, normal values, tables as well as key facts and principles provided during the exam. The goal is to move away from recall and rote learning. The more information provided, the better.

Students must be given the reference information before the exam so that they know that they won’t have to memorize these.

However, there should be no electronic devices or internet access allowed during the exam. No books or notebooks allowed either. Non-programmable calculators should be ok.

3 Easy questions first

Builds student confidence.

4 Number of questions per SAIL exam

A minimum number of questions is required for reliability.
Otherwise, the test is a hit-or-miss.

• Quiz: 20+ questions
• Midterm: 40+ questions
• Final exam: 100+ questions
• Certification: 200+ questions (high stakes, licensing exams)

Minimum of 20 questions per SAIL exam.

5 Visual information in most questions

Visual information promotes interpretation and application, avoids recall.

Include the following in most questions:

• Pictures, diagrams, graphs, charts
• Tables, data

6 Assess higher order thinking

Use rich case-scenarios with complex decision making to assess students’ ability to:

• Evaluate complex cases
• Apply learned knowledge to new scenarios
• Synthesize new information from available information (short-answer questions)

7 Students informed well ahead

Students should be informed about exam topics, preferably as book chapters (and not as learning objectives). An exam should be well-defined and limited. Students must be absolutely clear about the topics covered in the test. Note that a topic does not have to be taught to be included in an exam.

Also provide detailed information including number of questions, exam duration (time allotted, time per question), type of questions, exam marks as a percent of final course marks, location, timing, entry requirements, information that will be provided, what to bring, and other info.

8 Blueprinting to cover the curriculum.

All topics represented in exam questions (the blueprinting table).
This ensures validity.

Focus on important concepts in the course — avoid questions on trivia.
All questions reviewed by a colleague with 1:1 (not committee) friendly feedback.

9 Randomization (applies to electronic format, computer-based testing)

Question and answers sequences randomized student to student.
Non-essential details randomized slightly from student to student.

Computer automated checking for objective questions.

10 Objective vs subjective questions

90% objective questions (fill-in-the-blanks, MCQs, jumbled sequence)

10% subjective questions (short answer questions). These questions should be offered with a choice: ‘Answer 2 out of the following 3 questions’.

Avoid long essay questions (long essays can be home assignments with a rubric).

11 After the exam

Correct answers should be provided at the end of the exam even in summative assessments.

A survey form is given to students for their feedback and satisfaction ratings.

Optional but recommended for weekly quizzes, a post-exam discussion moderated by a faculty member, or a video recording shown explaining each question.

Psychometrics reviewed soon after the exam and before announcing results.
Questions revised based on psychometrics and student feedback for the next iteration.

12 Report card

Students provided a detailed report card.
Topic by topic feedback with a visual graph showing areas of high and low performance.
Class averages provided for overall score and for each topic.

Not to mention, SAIL stands for systematic assessment of internalized learning.