The Methods We Used to Research 1k+ Immigrants in NYC
There are over 3.1 million immigrants who live in New York, and Documented is a newsroom of fewer than 15 people with the goal of creating immigrant-centric news. We knew we wanted to publish multilingual journalism to reach multiple immigrant communities. But, we had to do the research to find out who our potential readers could be and what their thoughts are about local media. We needed to know: how do these immigrant groups get their news and how are current media offerings not serving them? These insights would help us design new ways of delivering journalism with our intended audience.
To be most effective, we had to narrow our goals to a specific immigrant group. Then, we analyzed public data to ensure our research wasn’t duplicative. We worked with community groups to design, test and distribute our surveys. After collecting and analyzing the responses, we dug down on key themes by hosting focus groups and one-on-one interviews.
Altogether, we spoke with over 1,000 local immigrants. Now we understand how to reach these new communities and what they want to know about life in New York.
We are sharing our research methodology with others to alleviate the barriers faced by these communities and support other newsrooms or service organizations as they pursue audience research.
Understand the sample you want to research on
As we prepared to expand our coverage to reach beyond Spanish-speaking Latino immigrants, we had to clarify our intended audience.
We learned the second and third largest immigrant communities in New York are Chinese immigrants and, combined, people from the countries of the Caribbean region that are not primarily Spanish speaking. But, no one had yet done a news assessment survey of these immigrant groups.
First of all, we asked: who are we going to serve? Is it Chinese immigrants or Chinese immigrants that live in New York? There are 13 countries that make up the Caribbean region. Where should we focus our attention? We funneled that prospective audience over and over until we had a clear plan.
Sample sizes
We know that our ideal audiences are:
Chinese immigrants who live in New York and don’t have English proficiency
Caribbean immigrants who live in New York and don’t have Spanish proficiency
However, without research or public data on our ideal audiences, we didn’t have an accurate number to have as a population size. Without it, we couldn’t calculate the sample size either.
For this reason, we decided to use the number of foreign-born Chinese and Caribbean population in New York that we found in public available data.
Foreign-born Chinese living in New York: 614,369
Foreign-born Caribbean living in New York: 476,939
Following these figures, we knew how many surveys we needed to distribute to minimize our margin of error. At the end of the survey taking period, we finished with these numbers:
Public available data
We wanted to avoid asking questions about matters that have already been covered in academic research, marketing research and the census.
For this reason, we collaborated with the Listening Post Collective (LPC) to make a sweep of all existing data about our prospective audiences.
We used these insights to design the questions that formed the two rounds of research we completed with each community.
Two rounds of research
After reviewing publicly available data and funneling down the community to the desired audience we want to serve, we found ourselves with so many questions.
For this reason, we decided to divide our efforts into two rounds of interviewing. As we were not paying every respondent, we wanted to keep the research process as short as possible.
Round 1: This is a short survey with around 10 questions, most of which are multiple choice. We created this on SurveyMonkey and distributed it digitially and with physical copies. The audience filled out the survey themselves and we entered them in a raffle for 10 $50 gift cards.
Round 2: This is a longer survey with 15 questions, most of which are open-ended. We created this on SurveyMonkey and Documented journalists filled it out based on interviews with audience members completed remotely or in-person. We offered 30 $25 gift cards.
How to get responses
Documented is a digital-first news source. We primarily publish on our website, documentedny.com, our newsletter for immigration professionals Early Arrival and in our WhatsApp community of Latino immigrants, most of whom are undocumented.
But, we haven’t yet built an audience of Chinese and Caribbean immigrants. So, to connect with these groups, we had to go to where they are — both online and offline.
We leveraged existing community pathways and sent our surveys to 54 local organizations that work with immigrants. They helped us design the survey and sent the collaborative version to their networks, which helped us garner responses and build trust with Caribbean and Chinese communities.
We published versions of the SurveyMonkey questionnaires in English, Chinese and Haitian Creole. To reach out to as many Caribbeans as possible, we launched a Facebook campaign that targeted ZIP codes where publicly available data showed large numbers of Caribbean immigrants were living. We also leveraged WeChat to distribute the Chinese survey to Chinese immigrants.
To reach immigrants who are not actively online, we printed cards with QR codes leading to the survey in different languages and distributed them, as well as printed versions of our questionnaires, to nonprofit organizations and at community events. Overall, the Documented team attended 24 community events during the research phase.
Digital challenges
As we mentioned before, we advertised that survey participants would be entered in a raffle to earn a gift card. To increase their chances of winning, people stuffed the Caribbean survey with multiple responses. We were able to isolate these by studying the IP addresses of submissions.
We noticed that the open-ended questions were sometimes hard for people with low literacy levels to answer.
It was difficult to get smaller groups within the broader Chinese and Caribbean communities to participate in this research. Many immigrants said they were reluctant to respond for fear of jeopardizing their immigration status. Young immigrants were mainly not interested in participating in the survey, and they are typically not as reliant on nonprofit community organizations, so they were more difficult to reach.
Our survey of Caribbean immigrants had a higher margin of error because the sample size was smaller.
About open-ended questions and print surveys
The few open-ended questions in the survey became very time consuming to analyze. Each qualitative response had many different data points we needed to codify and track.
In order to analyze this volume of responses, we needed to hire an external journalist with proficiency in Chinese language and data analysis, who helped Documented’s community correspondent.
Another challenged happened when we wanted to compare open-ended responses from the Caribbean and Chinese surveys. As both correspondents had different ways of codifying the answers, it became very difficult to compare.
We learned that when having open-ended questions, it is good to have a clear strategy on what you want to get from them, have uniformity on the question-making and the coding of the answers and an understanding that the analysis will take longer than with quantitative research.
On the other hand, the richness of the answers that people shared on the open-ended questions are now a valuable asset that Documented uses, and these insights helped us create the second round of our research, which used focus groups and one-on-one interviews.
Immigrant news findings
Overall, we interviewed 946 Chinese immigrants and 191 Caribbean immigrants. We combined our two rounds of research with existing data from census reports and other public findings to create a holistic picture of the news habits of these immigrant groups.
Some key findings from our original research include:
- 85% of Chinese New Yorkers who participated said they have seen information that they suspected to be false information or fake news when using social media like WeChat or Facebook.
- 77% of Chinese respondents don’t see themselves well represented in the news.
- 87% of Chinese respondents mention feeling “worried about the public safety in NYC” or feeling “unsafe” or that it is “dangerous” living in the city.
- 57% of Caribbean respondents said that the current media coverage is “too negative”
- 70% of Caribbean respondents use the apps “Nextdoor” or “Citizen” as a source for local news
These insights will inform news products we launch that will be designed specifically for Chinese and Caribbean immigrant communities.
Download our Chinese & Chinese Immigrant Communities Research
