Can we trust ourselves to be normal?

I remember writing the WAIS over the course of three days, in blocks of a few hours each. It was a long, boring, and sometimes frustrating process. The results weren’t unexpected- poor memory performance, but very high performance in cognitive flexibility and other executive function domains. It seems silly, but I never quite considered what exactly it meant to fall into the XXth percentile, until I learned that psychometric tests are built to facilitate comparisons to a norm. Naturally, the question arises- how are these norms established?

As a broke student, I often participate in studies for payment here at McGill. As a participant, I’ve contributed to the dataset of a wide variety of labs, some of whom were developing psychometric tests using this information (one memorable experience I’ve had in contributing to the development of a psychometric test was in Professor Dirks’ lab; it was my first experience with the rubber hand illusion, and felt shockingly weird). If the purpose of administering a test is to facilitate comparisons to a norm, why are so many labs on campus falling prey to convenient samples? Or are McGill students more “normal” than it seems?

I would argue that as a population, we are a group of intellectual and socioeconomic outliers. Less than half of McGill applicants receive an offer of admission, and according to the Deputy Provost in 2011, the Student Demographic Survey identified the student population as “… [coming] from relatively privileged backgrounds… [and] not as socioeconomically diverse as undergraduates at other Canadian universities” (McGill University, 2011). It has been well established that socioeconomic conditions is a hugely important deterministic factor in many facets throughout the course of a person’s life, including intellectual and psychological domains. If those establishing the norm live with a relative advantage, how fair is it to compare an entire population to a privileged few?

Investigating test-retest reliability is an important endeavour to undertake in order to combat this confound. However, if these investigations into validity are performed at these very same institutions that rely on convenience sampling (like McGill), can we really stand firm with the conclusion that psychometric tests facilitate valid comparisons to the population at large?

That’s not to say there isn’t value in a narrowly defined sample population, an approach which is found for the development of tests for a very specific population. Certain tests developed at McGill, such as the Montreal Cognitive Assessment (MoCA), rely on these patient populations in their development, and compare them to demographically-matched controls. This approach, to match potential contributing factors of a state or pathology across all the participating groups in the experimental design, is a valuable way to ensure that the test fulfills its purpose and enables valid comparisons of the results.

I understand that finding appropriate subjects is painful. A friend who recently graduated still hasn’t published her research, as she is still recruiting subjects! The nature of her work is such that she cannot look to McGill for participants, and the pains she goes through in recruiting participants is something that I wish she didn’t have to experience (to be fair, she’s running a longitudinal study of sex workers, and went into the project knowing the rate of attrition for this population would be discouragingly high). However, my friend seems to represent the opposite end of the spectrum, and it appears that there aren’t many labs whose recruitment efforts lie in the middle, where they make an attempt to reach out to recruit from a wider, non-McGill population. Perhaps nothing would change even if these efforts were made. But, knowing what we know about the impact of privilege and socioeconomic status, my rather non-specific hypothesis is that we would see a shift in the data, towards something that more truly reflects the development and experience of most members of our society, a more true “norm” that might just surprise us.

Bibliography
https://www.mcgill.ca/es/admissions-profile
https://www.mcgill.ca/studentlifeandlearning/files/studentlifeandlearning/final_report_1.pdf
http://onlinelibrary.wiley.com/doi/10.1111/j.1532-5415.2005.53221.x/pdf