The China Index: Crowdsourcing a global picture of PRC influence
Ttcat / Co-founder & CEO, Doublethink Lab
In 2019, I was invited to join a Southeast Asia digital rights camp, COCONET, in a faraway resort in the Philippines, a five-day un-conference style event with hundreds of activists, where nobody wore shoes. We all slept in villas and shared rooms. On the first morning, after the ice-breaking, attendees were asked to propose sessions that they’d like to host during the week. You had to write down the topic on A4 paper and pin it on a big wall, and do it quickly to secure a popular time slot. I proposed a session mapping China’s influence in Southeast Asia.
In the session, we sat inside a big wooden house with no chairs and people brainstormed every concern they had about China’s influence in their countries, using red, orange, and green stickers. Eventually, a few dozen people joined the mapping exercise, and it became a giant physical sheet on the wall for everyone to see.
It was a small crowdsourcing effort from activists in the region to document China’s activities in their own countries. We were surprised by how many similarities we hadn’t realized, how many different approaches appeared, and how many gaps there were indicating things we didn’t know yet. Activists shared even more concerns than those of us from Taiwan, regarding their own governments learning from and leaning towards the PRC’s authoritarian control and suppression tactics, for example.
Of course, none of us were qualified China experts from the academic world. A few attendees and I digitized our work into a spreadsheet. We used the session as a prototype to seek funding, find the experts, and expand the exercise into an Index project.
That’s where the China Index started. That’s how we pictured it at the beginning — and still do now — an open, crowdsourcing effort from people across countries, finding a systematic and standardized way to map, compare, and document People’s Republic of China (PRC)’s influence around the world.
Discussion: Designing the China Index
Our journey began with a basic question: Can we measure the PRC’s influence in the world? Capturing every aspect of PRC influence in a country is extremely challenging because influence is difficult to quantify, and because PRC influence operations are often under the surface and hard to observe.
In-depth case studies of PRC influence operations reveal both similarities and differences region to region, and are also hard to categorize and measure in a standardized way. One strategy or approach in a Southeast Asian country may not be applicable in Europe, and the effects may differ widely.
So we come back to a slightly different question: Is there a way to compare how much countries are influenced by the PRC? As a first experiment, regardless of the weight or significance, we put forward a simple statement that invites a factual, binary response:
Country A joined the Belt and Road Initiative (BRI) while country B did not, can we all agree that PRC influence in country A is undoubtedly greater than country B, in terms of this specific aspect?
Then we applied the same approach to a group of countries:
A group of countries that have reported technological development in sensitive sectors with PRC-affiliated companies would be more influenced by the PRC than another group of countries that have not, in terms of this specific aspect.
If we can agree on the two statements above, then we would likely agree that: Even if a BRI country experiences certain degrees of PRC influence, it doesn’t mean that non-BRI countries experience less economic influence by the PRC. So we need more such statements, or Indicators, to get enough data in a wide range of domains to have a full enough picture of PRC influence in a particular country.
This is how the methodology for the China Index was conceived: If we create a large-scale yes-and-no checklist based on factual Indicators for all countries in the world, then we should be able to capture enough data to discern comparable differences between countries and generate a world ranking of PRC’s influence based on how many “yes” responses a country receives.
The more Indicators we could put forward — thousands in an ideal world — the closer to reflecting the reality of PRC influence the rankings would come. But considering the implementation difficulties, effort and budget limitations, we decided this checklist would not exceed more than a hundred Indicators in the first iteration.
How to make the Indicators?
That limitation makes every single Indicator a valuable resource that requires careful selection and design. Our guiding concept was to make a list of the “Top 100 statements relating to PRC influence” at the time the Index was put together, so we could at least get a global perspective of how PRC influence works in the world.
So we turned to a group of widely respected professionals with expertise relating to PRC influence operations and strategies, and invited them to form an Index Committee, leveraging their knowledge to design our first set of Indicators. To capture various aspects of PRC influence, we asked the Index Committee to refine 11 of the most relevant, common, and diverse indicators within nine different Domains while avoiding weighting any particular aspect too heavily.
Relevant, Common, and Diverse: Each Indicator should focus on one issue that reflects the current trend of PRC influence, which is also widely observed across regions, and will not overlap between Indicators to avoid double counting.
For example, an Indicator asking whether there are PRC military aircraft threats to our national air defense might be crucial to Taiwan but not commonly observed in other countries, and therefore has to be excluded.
Of course, we do not intend to imply that 11 Indicators can capture all of the PRC’s influence in a certain Domain, but we believe that: If one country responds “yes” to all the Indicators in the Media domain, then we can have high confidence to claim that the PRC’s influence in this country’s media is greater than a country in which the answer was “yes” to just half the Indicators.
Our basic argument is: Foreign influence cannot be captured by any single Indicator of this nature. It’s a complex system that interweaves people, institutions, and interests, and is geared towards building ideology, dependency, and rulemaking alignments. If we could design a set of comprehensive, common, diverse, and relevant Indicators based on factual, observable instances of influence, we could use this “big checklist” to compare one country to another, and arrive at a global and regional overview.
Two basic challenges
We soon ran into two new questions as we went about determining the yes-and-no Indicators.
First, there are different degrees of influence across countries in the same Indicator: 10 Confucius Institutes in a small country is different to 10 in a large one in terms of scale, and the significance differs if they are placed in top universities versus less important ones. We had to have a means of allowing responses that take into account the local context.
Another example: The PRC has bribed politicians in the country. This Indicator should be treated differently if this politician is a prime minister versus a local city council member.
Thus, in some Indicators where a) we believe that the evidence is widely available, and b) they relate to phenomena in which scale and significance are important, we ask respondents to do an additional grading from 1 to 4 if the answer is Yes. The grading is based on a combination chart of scale and significance that they have to assess based on the local context in different countries. Yes, this requires a degree of subjectivity, but it provides a standardized way to grade, and narrows the scope of further debate to focus on only the scale and significance with supporting evidence, which is also based on local context.
Second, the nature of influence has complicated ebbs and flows.
A few Indicators were originally designed in a way that linked cause and effect: The PRC threatened to cut off the import of goods from my country over a foreign policy alignment with the Taiwan issue. In this case, the two phenomena — the PRC applying trade pressure and the country aligning with the PRC’s foreign policy over the Taiwan issue — could have both occurred but for non-related reasons.
However, the causation will never be the same. The PRC government could threaten or exercise leverage based on specific dependencies as a means of pressuring a country to align with its interests; there could be no clear evidence of threats or pressure but high exposure to PRC influence is observed; or policies can be changed without clear causation due to a friendly attitude in general.
Due to that consideration, we broke down all the Indicators that contain multiple instances of influence into individual Indicators: The Index inquires about one thing at a time. We then labeled the Indicators based on whether they fell into three Layers: Exposure, Pressure, and Effect for further data analysis.
Limitation on the data collection
Yes, there remains an element of subjectivity in the Index in the scale and significance judgment, as well as selecting the supporting evidence in each Indicator response (46 indicators are binary, and 53 are graded for scale and significance). In an ideal world, we would contract a statistically robust number of probability sampled experts to smooth out this subjectivity in all the countries, or ask all the country experts to do new research on all the Indicators. But practically, there aren’t enough experts we could reach out to in every country. In some relatively closed countries, it could be even harder to get more than a handful of willing and qualified respondents to work with.
Considering these challenges, we designed the methodology to balance the need to incorporate scale and significance within reasonable, practical limitations. We ask our regional partners to find qualified local experts who live in or have years of research expertise in the Index country — they could be China researchers, civil society organizations which understand the political context, or investigative journalists who have good experience in fact-checking sources of information. In addition to the respondents, all the answers will also be reviewed by at least one other local expert — and some countries did find more.
The requirements to provide evidence to support the judgments and making this evidence open to the public is also designed to facilitate auditing by everyone, so that we can take this feedback to improve the index. We believe this kind of transparency could help minimize bias to a reasonable degree.
I’d like to emphasize this: There is a real threat for researchers, activists, and NGOs to use their real names or affiliation in some countries, regardless of whether they fear PRC harassment or their authoritarian governments — lots of our Indicators reflect their own government’s lack of accountability or strong ties with the PRC — so we ask all of their consent to be affiliated with us. Only the regional partners, who coordinate the research, need to publicly share their names. That’s why you may find responses or reviews conducted by anonymous experts on our website. While we can’t reveal their identity, we can certainly confirm they are all legitimate experts from the country and qualified according to the criteria in our methodology.
Moving forward the Index will only improve
Almost 180 experts and partners from all over the world participated in this project during COVID. They are our advisory boards and consultants who debate the methodology back and forth with us; they are the Index Committee who co-worked on a huge online document and carefully polished the terms and language we used; they are our regional partners and local experts that spent weeks collecting data and writing the responses; they are our colleagues from Doublethink Lab who did the completeness checks for all the answers and evidence. The China Index is an amazing piece of collaborative work and we appreciate all the effort everyone put in to make it possible.
It’s not only an Index, but also a huge database for thousands of references to be documented. We hope this project will give civil society, journalists, researchers, activists, and policymakers a better understanding of the PRC’s influence in their respective countries and regions and create new research and actions to respond to it.
There might be some evidence available but missing in our Index — and we encourage participation from the public and research community in this regard — or some countries’ rankings may be underestimated due to lack of public information, or URLs may have expired or deadlinked before we could archive them. But overall, we are confident about the ranking results and proud of our three-year journey with this global community.
Our work on the next iteration of the China Index has already begun. We aim to update the Indicators to reflect what has changed since we began our work. We will also strive to improve the quality of responses and our data collection process. And like I said at the beginning, this is an open source project. We are open to corrections and feedback, and welcome more contributions and collaboration. Please get in touch.
