The Truth About GitHub’s Gender bias

Alice Marshall
May 20, 2016 · 4 min read
Image for post
Image for post

By now, you may have heard of the showing that women get their code accepted more than men, except when people know that it’s a woman. If you haven’t, feel free to look at these : , , , or one of the many other sources that have covered this. The story, though, when you actually read the research itself, is not quite as black and white as everyone is making it out to be. As someone who’s done a lot of regression analysis in my day, let me take a moment to clear up some confusion surrounding the actual results of this study.

Firstly, the most important result of this study, in my opinion, is that female users are more likely, on average, to get their changes on GitHub accepted than male users. This conclusion is based on the researcher’s use of a program to link 1.4 million GitHub user profiles with email addresses with Google plus profiles, where this group of users has self-identified their gender. Female users’ acceptance rate (or merge rate) is 78.6% and men’s is 74.6%, a difference that is statistically significant.

This result is really impressive when you consider the fact that women make up approximately while men make up around 88.8% (according to a 2013 study).

In addition to these general results, the paper further divides up the user data into “insiders: people who are owners or collaborators of a project” and “outsiders: everyone else.” Additionally, the researchers developed a novel, and seemingly solid, method of grouping users into those with “gender-neutral” profiles and those with “gendered” profiles.

What’s downplayed in the study is the fact that for insiders, when their gender is known, women still have a higher rate of acceptance than men. With gender-neutral profiles, women have a slightly lower rate of acceptance than men. It’s unclear whether or not these results are statistically significant or not.

For outsiders, both women and men’s rate of acceptance drops when their genders are known. To my dismay, the authors didn’t publish the results for men outsiders, which is highly problematic. However, I found the numbers in a article: “The acceptance rate for women coders was 71.8% when they used gender-neutral profiles, but dropped to 62.5% when their gender was identifiable. Men with gender-neutral profiles had an acceptance rate of around 69%, which declined to 63.3% for those whose gender was clear.”

The authors gloss over this similarity by saying “There is a similar drop for men, but the effect is not as strong” due to the fact that “women have a higher acceptance rate of pull requests overall, but when they’re outsiders and their gender is identifiable, they have a lower acceptance rate than men.”

Ok, so this paper hasn’t been peer-reviewed, which is the scientific method of maintaining quality, and it has some issues, which we’ve covered, but the authors have submitted it online for comments, which is great. There’s one last problem to resolve here. How do we know that the people reviewing the requests actually considered the person’s gender? Actually, we don’t. We can therefore not say for sure that these results were definitely bias-related. However, there’s still a troubling correlation here. For outsiders, women coders who use gender-neutral profiles get their changes accepted 2.8% more of the time than men with gender-neutral profiles, but when their gender is obvious, they get their changes accepted 0.8% less of the time.

In the interview with Nature, one of the researchers noted: “The difference is statistically significant, but whether the difference is substantial is another question that’s open for interpretation.”

Image for post
Image for post

The other thing to consider here is that there is a lot of (and I mean hundreds, if not thousands of academic articles), that has been peer-reviewed and accepted by the scientific community, showing that unconscious gender bias is extremely prevalent in our society, and it affects all of us profoundly. It affects both women and men. It affects us on an unconscious level — which means that you might consciously disagree with it. (“What? But I would never associate women with humanities and men with science?!”) We get it as a sort of unwanted present from all the millions of images and stories that we’ve been exposed to throughout our lives — on TV, in advertising, and in movies. No one’s immune to society-wide bias — you might have less of it in one area, but you’ll have more of it in another area.

For a bit of fun, check out for free to test your own biases — it takes about 10 minutes to complete. Our best tools regarding bias are to increase our awareness and try to create more structure that eliminates potential bias to the extent possible.

This study, despite its flaws, shows us some good things and some bad things. Female users are more likely to get their code approved on GitHub than male users. However, there is a correlation, when you’re an “outsider” and a female GitHub user with a profile that shows your gender, to having a tougher time getting your changes approved than a male user with a profile that shows he’s a man.

Gender Equality in Tech

Inspiration and knowledge about gender equality in tech.

Alice Marshall

Written by

Gender Equality in Tech

Inspiration and knowledge about gender equality in tech. Written by Alice Marshall, founder, Equality Inc., edited by Serge LaChapelle, site lead, Google Sthlm. Originally posted on the Swedish tech newspaper NyTeknik. Sponsored by Google. http://www.nyteknik.se/bloggar/equality/

Alice Marshall

Written by

Gender Equality in Tech

Inspiration and knowledge about gender equality in tech. Written by Alice Marshall, founder, Equality Inc., edited by Serge LaChapelle, site lead, Google Sthlm. Originally posted on the Swedish tech newspaper NyTeknik. Sponsored by Google. http://www.nyteknik.se/bloggar/equality/

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store