Two Assessors Are Better Than One

Whenever you are conducting a formal process of assessing an individual, whether it’s in a recruiting, selection, or training environment, you are most likely capturing subjective (based on your perceptions) performance ratings on that individual. While identifying the right data to gather is important, I would argue that gathering accurate data is even more important.

I have written many blog posts on the importance of understanding biases and how they affect your ratings; however, one thing we have not yet touched on is how multiple assessors can improve the quality of the data you are collecting and, by extension, improve the quality of your assessment program.

If you have multiple assessors evaluating a single individual and all assessors evaluate an individual as “exceptional,” you can provide a stronger case that this individual demonstrated exceptional performance, compared to if that assessment came from a single rater. In the psychology world, this would be known as high inter-rater reliability.

What if the opposite happens? What if one assessor views that individual’s performance as “exceptional” and another assessor views that candidate as “poor”? This is also important to note. Was the inconsistency due to assessor characteristics (e.g., biases) instead of that individual’s performance? If you can be certain that this difference was not due to assessor biases, then each assessor may have seen something that caused their ratings to be dissimilar. This gets at a truer assessment of performance and also prompts a conversation that allows people to discuss what was noticed by several assessors.

Neither of these situations can occur without multiple assessors. Having multiple assessors can allow for:

1. A stronger case on an individual

2. Multiple perspectives and vantage points to be taken into consideration

3. Identifying potential biases influencing the ratings

Next time you only have one assessor, consider if you may be missing out on one or more of these advantages to your assessment program.