Working on WJP’s Qualified Respondents Questionnaire team
Throughout my time in high school and college, I’ve frequently turned to statistics when writing research papers, from the results of Pew public opinion polls to reports from the World Health Organization’s data repository to the Mexican government’s household surveys on literacy. In the five weeks that I’ve been at the World Justice Project, I’ve gotten a better understanding of the work that goes on to prepare these indexes. While large organizations like the Brookings Institution or Gallup employ hundreds of researchers to conduct and analyze survey data, the World Justice Project generates an index quantifying rule of law in 113 countries around the world with a small staff of around 30. With polling partners around the world starting full fieldwork and the deadline for the final index coming up in December, each day at the office is understandably busy. The index is based on polling results from two groups of people: 1000 randomly selected citizens from each country, as well as at least 4 expert respondents each (usually doctors or lawyers) in the fields of public health, civil law, labor law, and commercial law.
I’m officially on the Qualified Respondents Questionnaire team, which handles the data from expert respondents, but during times of survey-related crisis, which arise frequently with a small survey team, I’ve also helped the General Population Poll team with the survey they administer to regular citizens. Although our D.C. office only has English, Spanish, and French speakers, the survey is administered in the official language of each country in which we poll. Local translators create a version of our master survey in the appropriate language, and it’s often my job to double-check that the translation doesn’t omit phrases — or even simply change the connotation or formality of a word. Since results from surveys will be compared across countries, it’s important that the questions be asked as similarly as possible. Achieving this often involves rather philosophical questions about the meaning of English words. For example, our English survey asks whether any household members have been murdered in the last year, which was translated by our partners in Russia as “killed in the last year.” Although being killed and being murdered are legally very different in English, the Russian language only has one word for both concepts — убийство. We spent a long time as a team discussing what “murder” means both legally and colloquially to U.S. English speakers, and eventually were able to work with the Russian translator to develop a better translation that better captured the idea from the English language survey (Умышленно убивать for the Russian speakers out there!)
Much of the rest of my time is spent recruiting experts for the Qualified Respondents Questionnaire. Lawyers and doctors in countries like Australia and Spain have very well documented contact information, but recruiting qualified respondents in countries with little access to the internet can be difficult. Generally, francophone Africa and the Commonwealth Caribbean are the most challenging regions in which to locate respondents — I’ve spent the past few weeks emailing and calling hundreds of these regions’ experts with very limited success. Still, we can see the surveys coming in in realtime from our desktop computers, and the little celebrations we have at the intern station each time a new St. Kitts and Nevis or Senegal questionnaire is completed makes the process worth it. I’m looking forward to seeing the results as the survey closes in early September!
Written by Katie Welgan’18, Chemistry major, FSI Global Policy Intern at The World Justice Project.