Tester’s Remorse

What happens when you write a psychological test assessing a specific construct, only to find that you score the worst at what it assesses?

In the scope of a class entitled “Psychological Tests”, I have been given the opportunity to finally write a cool assignment, if you will. Not unlike most students — I assume — and following years of undergraduate courses in psychology, I was finally able to build my own psychological test and take a slice out of the elusive pie that is research. But like any other scholastic fantasy — or pie for that matter — , it wasn’t as great as what I had thought it would be. I ended up learning things about my subjects that were far beyond expectation, and even more surprising, I learnt something about myself…

My grades are below average.

Now bear with me, I am not complaining about this fact, nor am I upset. What was so shocking about this information is that it was not what I had set out to find. As a matter of fact the questionnaire item probing participants for their GPA was simply there for gauging differences in procrastination behaviours — the construct I had set out to assess. Perhaps what really took me aback was the frequency distribution of my participants’ GPAs, displaying 81.4% of students having a GPA higher than 2.8 (which is considered a B letter grade at McGill University).

Why is this shocking, you ask?
Because I have never had a GPA much higher than 2.8.

…

Though this grade gap may in fact be a reflection of my procrastination tendencies (if so, yay!), — which I had acknowledged early on in my research as a factor — the issue was in fact that in my overwhelming stupor, I realized that testing a construct so close to your own habits might be unwise.

Being surprised by unsuspected results is generally an acceptable event in fields of research.
Being offended by the results of your study, is not.

The reason I bring this up is that we consistently wear our “unbiased glasses” when combing through research papers, without realizing that our own research might be flawed in the same way we expect others’ to be.
When we take these glasses off for a second, we open our eyes to the fact that test constructers are just human, and are prone to the same type of bias — especially when the data you collect does not confirm your ideas about YOURSELF and about what YOU know is true.

Following the comparisons that researchers and test constructers may make after viewing a summary of their data, it would then be likely that they feel what I will call a “Tester’s Remorse”. This remorse would stem from a tester regretting his or her decision to assess a construct that affects their daily life substantially — i.e. a procrastinating university student assessing procrastination in university students.

The beauty of the recognition of my own bias was that it was not visible while I constructed the questionnaire items, but rather after I took a good long look at the results, and reality set in.

I don’t know everything. And that, I now know.