Wikipedia
Invisible in the Hyperlink Network
One Hundred Weeks of the Wednesday Index
Two years ago, I started the Wednesday Index, which uses PAC’s Wikidata tool to measure the gender diversity in the biographies linked from 26 English Wikipedia pages — from ‘Reality’ to ‘Universe’, ‘Science’ to ‘Justice’ — to get a sense for both the extent of citation bias on Wikipedia and how quickly it changes. When I started this project, women represented less than 5 percent of the 2957 linked biographies on this set of pages, a striking gender imbalance that’s both well documented and particularly concerning as it’s amplified elsewhere.
“Structural properties impact how visible and reachable articles about notable men and women are, since users and algorithms rely on this information when navigating Wikipedia or when assessing the relevance of content within a certain context,” Wagner, et al write. “For instance, search result rankings are often informed by centrality measures such as PageRank.” PageRank, which is a measure of the number and quality of links to a page, is often used as a proxy for importance.
Some Wikipedia articles, known as ‘orphans,’ are linked to by no other Wikipedia page. Arora Et al refer to such articles as “the dark matter of Wikipedia” and found that orphan articles also “encode structural biases: biography articles about women are substantially more common among orphans than expected from their overall frequency.” The articles exist, but in the shadows where no one can stumble across them.
Wikipedia has a goal of eliminating the gender gap by 2030. Over the past two years, how far has the encyclopedia come in closing its structural gender gap? I know of no comprehensive study, but the Wednesday Index provides one window into both the current extent of link bias and how it could change. Women now represent 245 of the 3107 linked biographies in this set of pages, for a total share of 7.9 percent.
Much of the progress towards equity is due to targeted work of David Palfrey and Women in Red. Palfrey writes:
In July 2022 I looked at these pages in order to see what could be done. This undermines the representativeness of the Wednesday Index, but concentrating on these reported pages will I hope provide visibility on the difficulties of addressing this systematic issue.
What the Wednesday Index represents now is the progress that’s been made with targeted effort, as well as the difficulties documented by Palfrey on the page he dedicated to the work. This essay is one in a series of pieces I’ve written over the past two years to raise awareness of the issue, the challenges, and the work being done (or that could be done).
All of the data visualization for the Wednesday Index can be viewed here, but I highlight a few findings below.
The Curious Case of Philosophy
In general, the total number of links on the Wednesday Index’s pages remained constant or increased, but the Philosophy page was unusual. In the figure below, one can see how page revisions drastically affected the number of internal links. In this case, the number of links to women’s biographies went from 9 at the start of this project to 1 (currently, the only woman linked is Susan Haack, the author of a referenced text).
The revision involved, among many other changes, removing a section dedicated to women in philosophy, which linked to Hipparchia of Maroneia, Arete of Cyrene, G.E.M. Anscombe, Hannah Arendt, Simone de Beauvoir, and Susanne Langer. Accompanying the section was a portrait of Mary Wollstonecraft, as well as a link to her Wikipedia page.
As of this writing, no images of women appear on the Philosophy page. Although the intent of the page revision was not to render women invisible, this is one of the fall outs.
Junyi Tao, who has studied how women philosophers are represented on Wikipedia, writes:
If marginalized philosophers’ pages are not connected to others, even though they should be and could be, their existence becomes invisible to Wikipedia readers, despite the fact that they have their own pages within the encyclopedia.
Tao found that as of May 28, 2022, “there are 261 women philosophers listed on the ‘list of women philosophers’ page, but only 49 of them are included in the general list of ‘Philosophers’ … 81% of the women philosophers listed on the ‘list of women philosophers’ page are excluded from the narrative of ‘who is counted as a philosopher.’”
The Growth of Mathematics
Over the course of two years, the number of links on the Mathematics page showed substantial growth, and the share of links to women’s biographies increased as well — from 5 of 54 total links in November of 2021 to 16 of 126 two years later (representing a share of 9 percent and 12.7 percent respectively). The bulk of these links appear in the article’s reference section, linking notable authors of cited works to their respective Wikipedia pages.
An increase in the number of linked biographies, accompanied by an increase in the share of links to women’s biographies, was the dominant trend for the Wednesday Index pages: Universe, which had 2 of 98 links to women’s biographies in 2021, now has 12 of 115.
Political History of the world went from 1 of 87 links to women, to 9 of 138 in November 2023. Although links to women’s biographies continue to represent a small share of the total (less than 7 percent), the trend is toward equity, a much needed correction to the very deep problems Samuel Baltz studied and documented for Wikipedia’s political science pages.
Medicine also shows growth in the share of links to women. Although the total number of links on the page remained fairly constant, the share of links to women’s biographies rose from 5 percent to just under 9.
Note that the Medicine article repeatedly uses the word ‘father’, as in ‘father of medicine’, a gendered language of recognition.
“The way Wikipedia’s contributors think can influence how they describe individuals resulting in a bias based on gender. We use a machine learning model to prove that there is a difference in how women and men are portrayed on Wikipedia.” — Natalie Bolón Brun, et al
No Change in the Light
The underrepresentation of the links to women’s biographies on pages such as Light has hardly changed. Women represented 1 in 91 links in November of 2021, and now represent 1 in 92 links. The page receives half a million views a year, but the share of links to women’s biographies remains at about 1 percent.
Similarly, the share of women on the Peace article (11 of 277 in 2021, and 10 in 270 in 2023) showed no growth, rather a small decline.
“hyperlinking policies assume that relevant information will be cited and linked regardless of gender. They do not take into account that studies show men cite men more often than they cite women, and men dominate the Wikipedia editing space” — Colleen D. Hartung
Final thoughts
The progress towards correcting the structural bias in Wikipedia’s hyperlink network is visible in the data I’ve collected over the past two years, but is also the product of a deliberate, targeted intervention on this set of pages.
Barriers such as the underrepresentation of women’s biography pages, as well as the underrepresentation of citations to women’s scholarly work, impede progress toward correcting the structural bias, as do the absence of tools that could make the imbalance visible and/or help editors correct it.
“For those working to advance academic equity through supporting citation diversity, it is important for everyone working in the system to be more reflexive about their own practice, rather than only women or those who identify as non-binary. Bricks ‘do not just fall into place; they are placed’. Anticipating the effects of citation imbalance and choosing to cite differently is part of what an ethical and deliberative scholarly practice should entail.” — Heather Ford, et al
To reiterate Ford: it is important for everyone working in the system to be more reflexive about their own practice. Correcting Wikipedia’s structural bias will not happen without awareness and effort on the part of the community, and without tools that will help make visible and correct the imbalances.
Read more:
References
Arora, Akhil, Robert West, and Martin Gerlach. “Orphan Articles: The Dark Matter of Wikipedia.” arXiv preprint arXiv:2306.03940 (2023).
Baltz, Samuel. “Reducing Bias in Wikipedia’s Coverage of Political Scientists.” PS: Political Science & Politics 55, no. 2 (2022): 439–444.
Brun, Natalie Bolón, Sofia Kypraiou, Natalia Gullón Altés, and Irene Petlacalco Barrios. “Wikigender: A Machine Learning Model to Detect Gender Bias in Wikipedia.” arXiv preprint arXiv:2211.07520 (2022).
Ford, Heather, Tamson Pietsch, Kelly Tall. “Gender and the invisibility of care on Wikipedia.” Big Data & Society, 10(2).
Hartung, Colleen. Challenging bias against women academics in religion. Vol. 2. Atla Open Press, 2021.
Langrock, Isabelle, and Sandra González-Bailón. “The Gender Divide in Wikipedia: Quantifying and Assessing the Impact of Two Feminist Interventions.” Journal of Communication 72, no. 3 (2022): 297–321.
Luyt, Brendan. “The inclusivity of Wikipedia and the drawing of expert boundaries: An examination of talk pages and reference lists.” Journal of the American Society for Information Science and Technology 63, no. 9 (2012): 1868–1878.
Luyt, Brendan. “Representation and the problem of bibliographic imagination on Wikipedia.” Journal of Documentation (2021).
OpenSexism. “Erasing Her from HisStory” https://medium.com/@OpenSexism/erasing-her-from-history-a5be2cdbe45c
OpenSexism. “The Wednesday Index: One Year of Gender Diversity Data Visualized” https://medium.com/p/a6458b94d52b
PAC2, Gender diversity tool:
https://www.wikidata.org/wiki/User:PAC2/Gender_diversity
PAC2, Wednesday Index visualization tool: https://observablehq.com/d/632d9e0eccea271d
Palfrey, David. “Gendered link bias” en.wikipedia.org/wiki/User:Dsp13/Gendered_link_bias
Tao, Junyi. “Visualizing missing voices in the history of philosophy: A network analysis of philosophers on Wikipedia.”
Venus, Nicole. “The Representation of Female Economists on Wikipedia.” Available at SSRN 4540744 (2023).
Wagner, Claudia, Eduardo Graells-Garrido, David Garcia, and Filippo Menczer. “Women through the glass ceiling: gender asymmetries in Wikipedia.” EPJ Data Science 5 (2016): 1–24.
Wikipedia 2030 Movement Strategy: https://meta.wikimedia.org/wiki/File:Wikimedia_2030_Movement_Strategy_Recommendations_in_English.pdf
Zheng, Xiang, Jiajing Chen, Erjia Yan, and Chaoqun Ni. “Gender and country biases in Wikipedia citations to scholarly publications.” Journal of the Association for Information Science and Technology (2022).
