Reliability: Is the Test Unreliable or Ourselves?
For the last entry of my blog, I wanted to look at unreliability of tests and how it can potentially be improved. The issue of reliability and unreliability discussed in class and the text book often attributed to the fact that the test did not have internal consistency, test-retest reliability and other draw backs of the test itself. Nevertheless, can the same test be reliable upon the testing of one group and less consistent in another group? Personality theorists like Walter Mischel found that the personality ceiling was .30, which indicates that personalities are more or less reliable in the first place. Therefore, it poses a problem to all the inventories trying to assess stable personalities. Nonetheless, it is not in the test per se that unreliability is found, but the construct of personality itself.
Based on Daryl Bem’s moderator approach findings, we can safely assume that all traits are not equally relevant to people. Furthermore, Mark Snyder’s Self-Monitoring theory also adds on to the whole issue of test reliability by proposing that people show different levels of consistency across space and individuals.
Indeed, through Bem’s 1974’s study on friendliness with a self-report inventory, he found that using the same test, some people were as unreliable as the personality ceiling indicated (correlations between .21 and .39 with different self-reports of the respondents, peers, families, etc.). However, those who thought that the trait was relevant to their self-image went above the ceiling of 0.30 to reach correlations of 0.55–0.57. The findings can be attributed to the fact that people value the trait of friendliness more than others; therefore, it takes on more relevance in their overall personalities.
Zuckerman and Koestner also found similar results in their 1989 studies. By measuring extraversion, they found that people who showed personal relevance to the trait of being an extrovert were found to be more reliable than those who did not think that extroversion was a trait that was as representative to their overall personality. Again, it is not the test itself that poses problem to reliability, but the difference between personal relevance to the trait.
Likewise, the Self-Monitoring theory stipulates that there are people whom on all traits, are consistent, which is yet another personality factor that might be taken into account when measuring traits. Based on the theory, people who score high on self-monitoring were inconsistent across different people and situations, thus being able to adapt to a wide range of situational and interpersonal demands. On the other hand, those who score low were those who were considered consistent in all traits dimensions.
The graph above shows that across different situational demands (whether to act in an introvert or extrovert fashion), high self-monitors are able to stretch more in contrast to their basepoint level (R), whereas the opposite is seen with low self-monitors. (Lippa 1976)
I think that the take-home message from these studies partly explains the personality ceiling of 0.30. While measuring a broad sample that is expected to be representative of the whole population, one does not escape the fact that both high and low self-monitors will be intertwined in the same sample thus creating a relatively lower reliability value than if they were separately sampled. By being able to find in the population those who are high self-monitors would mean that personality inventories can improve in their reliability. Also, it would mean that traits can be more consistent than traditionally theorized.
Bem, Daryl J.; Allen, Andrea, On predicting some of the people some of the time: The search for cross-situational consistencies in behavior. Psychological Review, Vol 81(6), Nov 1974, 506–520. http://dx.doi.org/10.1037/h0037130
Koestner, R., Zuckerman, M., & Koestner, J. (1989). Attributional focus of praise and children’s intrinsic motivation: The moderating role of gender. Personality and Social Psychology Bulletin, 15, 61–72.
Koestner, R. (2015). Are We Chameleons? Personal Collection of R. Koestner. McGill University, Montreal, QC.
