Multiple Mini Interviews: A New Frontier in Ability Testing
Kaplan, Princeton Review, Prep101 — with the rise of test prep companies and courses oriented on acing interviews, it is becoming increasingly difficult in modern day society to accurately assess an individual’s potential. Many of the current ability assessments (ex. MCAT) are too easily affected by prior knowledge about the test. Shortcuts that these test prep companies teach undermine the exam and end up favoring high SES individuals who are able to take them, essentially creating a pay to win situation. Recently in 2001, McMaster University set out to develop a new test to rectify these issues and better observe a person’s traits and abilities while making it harder to prepare for. The result of this endeavor was the Multiple Mini Interview (MMI).
Two prominent problems of traditional assessments were addressed during MMI development: inaccurate predictions of medical school performance, and poor ratings in interpersonal skills, professionalism, and ethics. Research into developing the MMI began in 2001 with an initial pilot round. 18 graduate students volunteered as “medial candidates” taking the interview. The pilot test received 0.81 reliability, an optimistic start for the innovative new project. In 2002 they expanded the study to include real medical candidates. After their standard interviews, medical students were able to volunteer to take the new MMI test format. Reliability remained high and in a subsequent follow-up of candidates that took the MMI, it was found that the MMI was the best predictor of clinical performance, professionalism, communication, and success at obtaining a license in comparison to interview methods available at that time.
The interview is structured into several short independent assessments, with each station timed for ~8 minutes. Each station can assess for any or multiple domains such as ethics, professionalism, interpersonal skills, management, etc. The stations are performed in rotation, with each candidate starting at a different station. An administrator is present at each station to rate the applicant’s performance and ensure that MMI is conducted fairly and on time. The obtained aggregate score reflects that person’s soft skills.
Since its introduction in 2001, the MMI is now used by dozens of medical, dental, and pharmacy schools to determine its student body. It has seen great results in predicting future outcomes and selecting qualified applicants. It has had high prediction of job performance and job retention. Some advantages include fewer resource requirements than standard interviews, low effect of security breaches on results, low influence of sex/status/minority background on results, reduced halo effect because there are multiple raters and stations, and importantly prep courses have not shown any effect on scores. One negative aspect that has been highlighted, though, is the fact that introversion has been seen to compromise performance on these interviews. Of course more research needs to be done on the validity and reliability of the test. Because validation has only been produced by McMaster and affiliated test companies, 3rd party research would be beneficial to ensure that the vested interests of the test makers does not skew data on test efficacy and validity. However, from a quick google search showing the low prevalence and scarce availability of prep courses for MMIs, it can be seen that the MMI is doing what it was designed for — you can’t pay to win.
