Tradeoffs for Donors Selecting Datasets to Measure Internet Censorship
By Terry Fletcher (Millennium Challenge Corporation)
This article is being censored right now. Depending on which country you visit this blog from, you may be reading these words, a fake error message, or a threatening warning from your government. The site hosting this blog, Medium.com, is currently censored in China, and has been reportedly blocked in Malaysia, Egypt, Albania, and Vietnam in the past. But if you are in one of these countries you might never know it. Even international donor organizations that want to promote a free and open internet can struggle to identify when and how a country is censoring its internet.
Censorship is inherently about the concealment of information, often including information regarding the censorship itself, and so is challenging to measure. While many components of human rights or democracy can only really be measured by some method of expert assessment, the nature of the internet means that censorship can also be measured by technological methods. In our paper, which was recently published open access in the Data & Policy journal at Cambridge University Press we explore the tradeoffs for donors when considering whether to use expert analysis or remote sensing to measure censorship of the internet.
How Do Governments Censor the Internet?
There are many ways that governments may choose to censor internet access. We focus on two: internet filtering where governments block specific sites or content, and internet shutdowns, where all internet access is cut off in a particular area for a given period of time. The prevalence of internet shutdowns has increased over time as security protocols, such as HTTPS, have made the filtering of specific content more challenging. While internet filtering targets the concepts or ideas that a government does not want its people to have access to, internet shutdowns often target the people that a government wants to not have access to the internet, sometimes targeting regions of a country or even specific neighborhoods. While some governments claim that they are justified in shutting down the internet to limit violent protests or instability, research has found that such shutdowns actually increase the likelihood of violence. Shutdowns are also particularly harmful as they impact the entire internet economy, not just controversial topics.
How Can You Tell If Your Internet is Being Censored?
Internet censorship is measured primarily in two ways, using expert analysis (where one or more experts are surveyed to determine the prevalence of internet censorship in a given country) or remote sensing (where technological means are used to detect specific instances of censorship). Within each of these methods there are a variety of specific methodologies. For example, Freedom House and the Varieties of Democracy Institute (V-Dem) both use expert analysis, but Freedom House trains a single expert in their methodology to complete a survey and issue a report, while V-Dem surveys hundreds of experts and then uses statistical methods to aggregate and normalize those results into a single score.
For our research, we look at four sources, two using remote sensing (Access Now and the Open Net Initiative (ONI)) and two using expert analysis (Freedom House and V-Dem). These datasets are some of the most widely cited in the literature, and three of them (Freedom House, V-Dem, and Access Now) produce annually updated data on the extent of internet censorship in a country. ONI produced annual data in the past, but does not any longer. We focus on Access Now and ONI, as remote sensing datasets that have manual oversight, i.e. instances of censorship are detected with technological means and then confirmed through volunteers, statements from authorities or telecom providers. We compare whether these datasets agree or disagree as to whether a given country is censoring the internet in any given year.
Which Measure Should You Use?
Overall, we find that remote censorship is more prone to false negatives, (failing to identify censorship when it exists). This is particularly the case in areas of the world with especially repressive restrictions on a free press (such as Central Asia and the Middle East) and may be due to the inability to manually confirm instances of censorship detected through technological means. Expert analysis on the other hand is more prone to false positives (identifying censorship when it does not exist). This may have to do with expert analyses conflating different types of limitations on free expression with internet censorship. This can be seen as V-Dem shows that internet shutdowns were almost as common in 2000 as they are today while most sources find that shutdowns were very rare before 2011, and much more common today. This may be due to V-Dem conflating other types of censorship. Freedom House has a similar potential conflation in the Philippines in 2020, where it was the only source to identify an internet shutdown, but other sources identify a shutdown of a major television broadcaster, not the internet in that year.
Different datasets will be useful to different stakeholders. Donors that value verifiability of data on censorship may prefer using remote sensing methods as these have clear empirical evidence when censorship occurs, making it harder for country governments to criticize the data as biased or unfair. Advocacy organizations that value completeness, and ensuring that especially restrictive governments are not able to avoid detection by restricting the press may prefer expert assessments. Everyday users, on the other hand may value real-time remote sensing tools with no oversight such as NetBlocks or the Open Observatory of Network Interference, to determine if the internet they are accessing is being censored right now.
I don’t know if you were able to read this article, or if it was censored where you are, but hopefully, if you made it through you have some tools to be able to tell in the future who is censoring the internet and where.
Related Research Article (Open Access)
Fletcher, T., & Hayes-Birchler, A. (2023). Is remote measurement a better assessment of internet censorship than expert analysis? Analyzing tradeoffs for international donors and advocacy organizations of current data and methodologies. Data & Policy, 5, E9. doi:10.1017/dap.2023.5
About the Author
Terry Fletcher is Senior Policy Officer at Millennium Challenge Corporation in Washington D.C, a bilateral United States foreign aid agency established by the U.S. Congress in 2004. It is an independent agency separate from the State Department and USAID.
*****
This is the blog for Data & Policy (cambridge.org/dap), a peer-reviewed open access journal exploring the interface of data science and governance. Read on for five ways to contribute to Data & Policy.
