Rooting Out Systemic Bias in Neuroscience Publishing
An interdisciplinary research team has found statistical evidence of women being under-cited in academic literature. They are now studying similar effects along racial lines.
By Izzy Lopez
Scientific papers are the backbone of a research community and the citation of those papers sparks conversation in a given field. This cycle of publication and citation leads to new knowledge, but what happens when implicit discrimination in a field leads to papers by minority scholars being cited less often than their counterparts? A new team of researchers has come together to ask this question and dig into the numbers of gender and racial bias in neuroscience.
The team members include physicist and neuroscientist Danielle Bassett, J. Peter Skirkanich Professor of Bioengineering at the University of Pennsylvania, with secondary appointments in the Departments of Neurology and Psychiatry in Penn’s Perelman School of Medicine, statistician Jordan Dworkin, then a graduate student in Penn Medicine’s Department of Biostatistics, Epidemiology and Bioinformatics, and ethicist Perry Zurn, an Assistant Professor of Philosophy at American University.
Their study on gender bias, which recently appeared in Nature Neuroscience, reports on the extent and drivers of gender imbalance in neuroscience reference lists. The team has also published a perspective paper in Neuron that makes practical recommendations for improving awareness of this issue and correcting for biases.
They are now working on a second study, led by Maxwell Bertolero, a postdoctoral researcher in Bassett’s lab, that considers the extent and drivers of racial imbalances in neuroscience reference lists.
Together, Bassett, Dworkin and Zurn are using their combined research strengths to uncover the under-citation of women or otherwise minority-led papers in neuroscience and to assess its significance. This research is fundamental in highlighting a true gap in representation in research paper citations, which can have detrimental effects for women and other minorities leading science. In addition, they provide actionable steps to address the problem and build a more equitable future.
Your research team is a distinctive one. How did you come together for a study about gender discrimination in neuroscience citations?
Jordan Dworkin: It was a fortunate coincidence. In the run-up to a big neuroscience conference, I started seeing discussions on Twitter about gender-based discrimination in neuroscience. There were stories being shared of women’s papers being overlooked and reviewers seeing reference lists that were almost entirely made up of men. It was illuminating, especially because some people in the discussion were hesitant to take those experiences at face value. This skepticism, and occasional combativeness, seemed to stem from the view that citations are an untouchable, scientific bastion where researchers’ decisions are fully objective. The tension between that view and scholars’ lived experiences encouraged me to explore the existing literature on this issue.
As it turns out, there is really strong literature on issues of diversity and citation in science. Some disciplines have done field-specific investigations, such as the foundational studies in political science, international relations, and economics, but there wasn’t yet any research in neuroscience. Since biomedical sciences often have different approaches to citation, it seemed that it would be worth doing a deeper neuroscience-specific investigation to give quantitative backing to the issue of gender bias in neuroscience research.
Danielle Bassett: When Jordan and I started working together on this project, I knew it was important. To do it right, we needed to present the information in a way that made it actionable, with clear recommendations about how each of us as scientists can help address the issue. We also needed to add someone to the team with expertise in gender theory and research ethics. We especially wanted to make sure we were discussing gender bias in a way that was informed by recent advances in gender studies. That’s when we brought Perry in.
Perry Zurn: I’m a philosopher by training, with a focus on ethics and politics. Citations are both an ethical and a political issue. Citations reflect whose questions and whose contributions are recognized as important in the scholarly conversation. As such, citations can either bring in marginalized voices, voices that have been historically excluded from a conversation, or they can simply replicate that exclusion. My own field of philosophy has just as much of a problem with gender and racial diversity as STEM fields, something Dani and I have been talking about for a long time. This work seemed like a natural point of collaboration.
Describe this study and what it means for promoting gender diversity in neuroscience.
Bassett: For years now, various scholars and activists in science have drawn attention to issues of gender and racial inequities in the field. Most of these conversations, however, have placed responsibility for change in the hands of people in power, such as journal editors, grant reviewers, department chairs, presidents of scientific societies, etc. But many of the imbalances people notice, whether in conversation with peers or through studies like ours, are perpetuated by researchers at all levels. Given that every research project is built on prior research, and therefore every paper has a reference list of citations, every researcher can make a difference. Who we choose to cite matters.
Dworkin: To understand the role of gender in citation practices, we looked at the authors and reference lists of articles published in five top neuroscience journals since 1995. We accounted for self-citations, and various potentially relevant characteristics of papers, and we found that women-led papers are under-cited relative to what would be expected if gender was not a consideration in citation behavior. Importantly, we also found that the under-citation of women-led papers is driven largely by the citation behavior of men-led teams. We also found that this trend is getting worse over time, because the field is getting more diverse while citation rates are generally staying the same.
For a very simple example, if there were 10 women and 90 men neuroscientists in 1980, then citing 10% women would be roughly proportional. But with a diversifying field, say there are now 200 women and 200 men neuroscientists and citations are still 10% women. Sure, the percentage of women cited didn’t go down, but that percentage is now vastly lower than the true percentage in the field. That’s a dramatic example, but it shows you that if we’re going to call for equality in scientific citation, the number of women-led papers on a given reference list should reflect, or even exceed, the number of available and relevant women-led papers in a field, and our work found that it does not.
Bassett: This under-citation of women scientists is a key issue because the gaps in the amount of engagement that women’s work receives could have detrimental downstream effects on conference invitations, grant and fellowship awards, tenure and promotion, inclusion in syllabi, and even student evaluations. As a result, understanding and eliminating gender bias in citation practices is vital for addressing gender imbalances in a field.
Why are citations important to gender representation in neuroscience?
Dworkin: Unlike hiring and grant funding, citations are something every researcher participates in. For example, as a graduate student I did not have any role on a faculty search committee, or any power in an academic society to decide on conference speakers, but I still have reference lists in all my papers. Citations are a unique area where all researchers play a direct role, where each person has a chance to reflect on their own practices and use those practices to create change in their field. Their ubiquity means that citations function as a conversation within a field, and their presence or absence can signal whose work is valued and whose is not. On a more concrete level, citations are often used as metrics for a variety of important, potentially career-defining, decisions.
Bassett: There are a lot of underrepresented scholars who have fantastic ideas and write really interesting papers but they’re not being acknowledged — and cited — in the way they deserve. And there are great role models for all the young women who are thinking about going into science, but unless the older women scientists are being cited, the younger ones will never see them. Without serious changes in the field, and a deep commitment to gender and racial diversity, many young women and minority scientists won’t stick with it, they won’t be hired, they won’t be promoted, and they won’t be put in the textbooks.
Zurn: Exactly. I think it’s important not only to think about who we’re citing as leading scientists, but also what sorts of people we’re representing as scientists at all. If you are looking at neuroscience as a field and you see predominantly white cisgender men in the research labs and the reference lists, then you begin to think that is what a neuroscientist looks like. But this homogeneity is neither representative of an increasingly diverse field like neuroscience, nor supportive of continuing efforts to diversify STEM in general. We need to expand what a scientist looks like and citations are one way to do that.
When you conducted this research, how did you “know” the gender of authors on a reference list?
Dworkin: In this work, we use a software package called Gender API that combines data from social media and government databases to predict someone’s likely gender from their first name. Though algorithms are useful when working with tens of thousands of author names, they are also a flawed way to assess something as complex and personal as gender. Perry’s expertise really helped us understand this complexity and approach our measurement in a thoughtful way.
Zurn: It is important to recognize that a name is not a gender, but names are gendered. In this study, we were working with data that probabilistically assigns a gender to a name based on social media profiles as well as with the Social Security baby names database, which records names given to babies assigned either female or male at birth. Those assignments, and even those profiles, are not necessarily reflective of a person’s own gender, especially if that person is transgender, non-binary, and/or intersex. To some, that might feel like it softens the data, but I think it helps capture what happens with implicit bias. When we hear a name that we unconsciously interpret as masculine or feminine, we may, again unconsciously, think this person may be a better or worse scientist. Implicit bias against women scientists may be reflected in lower citations for people with names more typical of women.
Bassett: To be clear, we are also really eager to develop additional ways to understand the specific biases faced by transgender and non-binary scientists.
What actionable measures can researchers take to increase the representation of women and other minority voices in their reference lists and in their field as a whole?
Zurn: There is a lot that each of us can do. Everyone, from faculty, to post docs, to undergraduate students can increasingly incorporate more women and minority voices in the field and break the cycles that continue to exclude them. We can consult diverse scholars as we conceptualize a new research project, and we can recommend others do the same when we review their papers. We can diversify our reference lists and include a citation diversity statement at the end of our published papers. This can prompt readers to consider their own work and ask, “Who am I citing? How diverse is my reference list?”. Through efforts like these, we are aiming for more than merely mechanical changes. We want to promote a culture of self-reflective scholars and scientists, who each do our part in creating change.
Bassett: And on top of all that, remember that we can cite people with our words, in conversation, online, and in the classroom. In the classroom, one can return to an underrepresented student’s point from earlier rather than deferring to the most popular speaker in the room. These are all ways to cite diverse scholars and promote the visibility of women and minority scientists in a field that all of us, including non-scientists, can participate in.
Zurn: With diversity work, people who get involved often think that in order for change to occur, something big has to happen on the national level or institutional level, and those things do need to happen. But I think it’s also important to be the kind of person in your everyday life who changes the profession one day at a time. Diversifying your citations is one way to be that change.
You have a similar study that examines systemic racial bias in citations in development. Can you share some of the preliminary findings, and how this compares and intersects with gender bias?
Bassett: We are indeed working on a second study that examines author race predicted from first and last name combinations. We hypothesize that people of color will be under-cited relative to the proportion of papers they author in the field, and furthermore that that under-citation will largely be driven by the citation practices of white authors. Which means us. We are scientists, and we see the evidence of pervasive racial bias, and we therefore — rather than being affronted at the insinuation of error — expect to find both gender and racial bias in ourselves. And we plan to address it in ourselves, with every paper we write, and every citation we add.
Danielle Bassett also has appointments in Penn Engineering’s Department of Electrical and Systems Engineering and Penn Arts & Sciences Department of Physics and Astronomy.
Jordan Dworkin is now an Assistant Professor of Clinical Biostatistics in the Department of Psychiatry at Columbia University.
Kristin Linn, Assistant Professor of Biostatics, Russell Shinohara, Associate Professor of Biostatistics, and Erin Teich, a postdoctoral researcher in Bassett’s lab, also contributed to the study published in Nature Neuroscience. It was supported by the National Institute of Neurological Disorders and Stroke through grants R01 NS085211 and R01 NS060910, the John D. and Catherine T. MacArthur Foundation, the Alfred P. Sloan Foundation, and the National Science Foundation through CAREER Award PHY-1554488.
