Measuring Personality in Developing Countries

Executive Summary

Many organizations use personality tests to select employees. The most commonly-accepted model for personality assessment — called the Big Five — has been shown to have good cross-cultural validity. The results of a recent study, however, suggest that commonly-used personality questions have much lower validity than previously thought for individuals with low levels of education from low-income countries. HR professionals should ensure that any personality assessment they use for employee selection in developing countries has been validated with the specific populations from among whom they hope to recruit.

Morgan Sailshade, Inc.* is a global manufacturing company headquartered in Milwaukee. Like most of their competitors, they do much of their manufacturing in the developing world, where labor costs are much lower. MSI has contracted Gale Systems — an assessment firm — to administer technical and personality assessments for employee selection at all their locations.

Gale’s personality assessments are based on the Big Five model, the most commonly-used personality model in the world. There’s a large body of research demonstrating that Big Five traits predict various kinds of job performance. Specifically, Gale has found that candidates with high levels of conscientiousness — one of the Big Five factors — tend to excel in the kinds of manufacturing roles that MSI is offering.

MSI piloted the assessment program at their sole U.S. manufacturing location and were pleased with the outcome. When they rolled out the program to their Kenyan factory, though, the results were disappointing. Gale responded by providing evidence that they had validated their assessment with a large, representative sample Kenya.

What went wrong?

One possible explanation is suggested by a recent study conducted by eight researchers from institutions around the world. Their analysis compared the results of 94,751 face-to-face personality assessment interviews with individuals from low-income countries with the results of the internet-based norming samples typically used in personality research. They found that some of the most common questions used to measure personality in the internet studies failed to replicate when administered in a face-to-face format with individuals from lower-income countries.

Although the researchers weren’t able to point to a single explanation for this discrepancy, they argued that individuals with less education tended to respond to personality questions differently than the highly-educated individuals who typically respond to internet-based surveys.

Gale’s norming sample, for instance, was probably conducted among younger Kenyan workers who tend to have higher levels of education than the older workers that are most attracted to positions with MSI. An assessment that works just fine for young, urban workers with college degrees might fail when used with less-educated middle-aged workers in rural areas.

Takeaways for the HR Professional

When using any assessment for employee selection, you should ensure the instrument has been validated with a sample that matches the population from which you’re recruiting. In many cases, assessment firms provide data at the country level, which might not be specific enough. If you plan to recruit employees with low levels of education, for example, ask your test provider to provide evidence that their assessment has been validated for this specific population.

Source

Laajaj, R., Macours, K., Hernandez, D. A. P., Arias, O., Gosling, S. D., Potter, J., … & Vakis, R. (2019). Challenges to Capture the Big Five Personality Traits in Non-WEIRD Populations. Science Advances.

* This is a fictional example for explanatory purposes. Any resemblance to an actual organization is purely coincidental.