Multiple Choices

Take your best guess…

Midterm season is coming to an end. Praise (insert omnipotent/all-powerful being of choice here)! With this begins a period filled with anxious waiting and self-doubt, characterized by an unhealthy obsession with the little red dot that lives in the upper right-hand corner of MyCourses. For few things motivate a struggling undergrad more than knowing they need a mere 119% final exam score to get that A.

But I digress...

The purpose of this post is to reflect on assessments of days gone past, specifically those of the multiple-choice variety (which I felt was appropriate given that we recently received grades for a multiple-choice test of our own).

I, for one, did not do as well as expected, though that may have had something to do with sleeping through my alarm and stumbling in late — a big thank-you to Kristin for accommodating me! That said, I felt the midterm was fair, well-written, clear, and overall representative of the material covered in class. However, contrast that with the ambiguously worded, two-of-the-above, none-of-the-above style abomination I suffered through this past Monday (little dramatic, I know) and one can start to see why many of us develop such strong feelings towards multiple-choice based assessments.

One common complaint is that multiple-choice items penalize creative examinees: “I’m a deep-thinker, a ruler of rhetoric, a master of the English language; my creativity is matched only by my pretension and this lowly multiple choice exam is a test of nothing more than superficial knowledge and rote memorization,” you mumble to yourself, as you begrudgingly bubble in answer choice C. Unfortunately, despite many of us believing this association to lie at the root of our troubles, a study by Powers and Kaufman (2004) found no evidence in support of it. The take-away was that creative students are not penalized by multiple-choice style exams, meaning that you probably should have just studied that little bit harder. Darn.

Point awarded to multiple-choice.

In fact, despite the love-hate relationship many of us seem to have with these tests, they do provide a number of benefits. This is perhaps not surprising given how much of testing, academic and otherwise, seems to revolve around the letters A, B, C and D. These benefits include increased sampling of the content domain, objective and efficient scoring, a reduction in the influence of construct-irrelevant factors, and finally, depending on your perspective, subjectivity to random guessing.

Two more points to multiple-choice.

Having recently read Chapter 7 of our text (which focuses on item analysis and the relationship between reliability/validity of scores and the quality of the test) I began to wonder how our midterm would fare when put under the microscope. What was the difficulty level of items on our exam? What about the ability of items to accurately discriminate between high and low-scoring examinees? The quality of distracters?

Personally, I think it would be neat for Dr. Stotland to spend a few minutes presenting this data when introducing the material (maybe tomorrow??). I suspect that this is particularly relevant given that we’ll be conducting similar analyses when writing our final papers. Anyone else? Thoughts?

Let me know in the comments below!

-260423700-

References:

Powers, Donald E., and James C. Kaufman. “Do standardized tests penalize deep-thinking, creative, or conscientious students? Some personality correlates of Graduate Record Examinations test scores.” Intelligence 32.2 (2004): 145–153.

Reynolds, Cecil R., and Ronald B. Livingstone. Mastering Modern Psychological Testing: Theory & Methods (2012).