A recent article on Quartz posed a neat little question: “Is it possible to determine whether a twitter user is pro-choice or pro-life without reading their twitter profile?”
Through lots of very pretty charts clearly establishing the author’s extensive expertise in the creation of pretty charts, the article eventually concluded “Yes. It is.”
Personally, I had some trouble convincing myself this didn’t amount to a silly game played in a sandbox of sample-bias. Perhaps I was distracted by the charts (which, again, really great job on those), or perhaps the author was just a bit off the mark. I gave it a second read-through, following along carefully to assess the reasoning and be absolutely sure the charts were in fact pretty. All was good until I hit this bit of text:
90% of users with strong views in the Gamergate and conservative communities are pro-life;
Woah woah woah. Now hang on a second, buddy. Friend. Comrade. Unspecified-relation-with-whom-I-should-schedule-an-intervention-to-address-the-tendency-for-misleading-label-usage. You are tearing this family apart.
What are we talking about here? Is it “90% of the group labeled GamerGate and Conservatives are pro-life”
Or you saying:
“90% of the group labeled GamerGate is pro-life — as is 90% of the group labeled Conservatives.”
Or:
“90% of the group labeled GamerGate is pro-life. But this label includes only the people who tweeted under #ShoutYourAbortion, and not the people in their extended network who also identify as GamerGate.”
Because, if you’re saying the last thing, then the GamerGate label isn’t generally predictive of views on abortion. If you’re saying 90% of GamerGate members are pro-life — that sounds like the exact opposite of the GamerGate community I know. And if you’re saying that an aggregate grouping of Gamergate members and Conservative twitter users is 90% pro-life: then I very strongly suspect that the 10% of that group which is not pro-life is almost indistinguishable from the portion of the group composed of GamerGate members.
But suspecting is for wusses who fear closure. We’re here to take chances, make clarifications, and get messy.
Two days ago I sent out this tweet:
Now that some time has passed and exponential growth has had a chance to do its thing, it’s time to do some fucking science.
Are you pumped? YOU BETTER BE PUMPED. BECAUSE IT’S TIME TO SPEND 5 HOURS MANUALLY TABULATING INDIVIDUAL RESPONSES INTO A SPREADSHEET WITH THE CAREFUL CONSIDERATION AND RESPECT EACH ONE DESERVES.
LET’S FUCKING DO THIS.
Quick Aside about Methodology: (you can skip this if you don’t care)
Each response is ranked on a scale of Pro-choice to Pro-life, and a scale of apathetic to ambivalent. Also included is a measure of ambiguity (to easily filter out responses below some threshold of certainty about the respondent’s intent).
In lieu of a computer that can pass the Turing Test, rankings were scored by manually reading each tweet to minimize room for error.
A user ranks higher on the Pro-Choice axis if they list themselves as Pro-Choice without mentioning caveats. So, a user who says “Pro-choice” ranks higher than a user who says “Pro-choice, but only before the third trimester.”
Users who voice anything of the form “Pro-choice, but only in cases of rape” are ranked as strongly “Pro-life,” despite how their response was phrased.
Ambiguity is ranked on a scale of 0 to 4. With 0 being “absolutely clear” and 4 being “I have next to no clue what the fuck this person is trying to say.”
Anything with an ambiguity level of 2 or less is very close to guaranteed to be on the correct “side.” That is, there is almost no risk that a respondent who ranks on the Pro-Choice side of the scale with an ambiguity <= 2 might actually be pro-life, and almost no risk that a respondent ranked as pro-life at an ambiguity level <= 2 might actually be pro-choice.
Each ranking includes the user’s twitter handle, both for verification and so that people might make pretty connectivity charts between users if they are so inclined. (Please someone? Please make the charts?)
So that people don’t complain, the graph and numbers below are filtered to include only respondents with an ambiguity level <=2. However, it should be noted that because of the large sample size, we could set the ambiguity level to whatever-the-fuck and it hardly makes a dent in the numbers. The raw sample comprises 486 respondents. The filtered sample is the 93% of respondents with non-ambiguous answers.
Respondents who rank very weakly in either direction are labeled as apathetic/ambivalent. After filtering for ambiguity, there are no respondents who rank very weakly pro-life, so relative size of the pro-choice group would be slightly larger if we threw the weakly opined respondents into their respective binary-dichotomy buckets.
Results
Okay, so the Quartz piece seems to imply 90% of GamerGate members are pro-life. Survey says:
80.3% Pro-Choice, and 16.5% Pro-Life.
Which means the Quartz article wasn’t just giving the wrong impression, but the results are almost the exact opposite of what it implied.
But “the author could have just been grouping GamerGate and Conservatives” you respond. And this is true. The author could have been doing that. But if so — why the fuck?
Let’s put the decision in perspective real quick.
According to Gallup polls, 50% of Americans identify as “Pro-Choice”
Which means GamerGate members are 30 percentage points more likely to identify as “Pro-Choice” than the average American adult. And honestly this is totally unsurprising in light of previous surveys on GamerGate’s demographics.
But hang on, it doesn’t end there. Because Gallup polls also give us a breakdown of Pro-Choice identification between Republicans and Democrats, we can compare GamerGate members to the average American liberal.
Soak that in for a second. 68% of Democrats identify as Pro-Choice. Which means a randomly selected GamerGate member is 12% more likely than a randomly selected Democrat to identify as Pro-Choice.
So if the author is “just grouping” the two — he is literally grouping conservatives with a bunch of people who are more liberal than the average liberal as measured by the very issue in question.
What?
How?
Why?
I mean, don’t get me wrong, I fully recognize that giant armies of rabid conservative internet misogynists make for some DAMN SEXY stories. Hell, feminists comprise a demographic large enough that there are entire sites whose bread-and-butter consists almost entirely of trolling them (*ahem* Jezebel). I can totally see the temptation to opt for this sort of thing.
But really?
Really?
You know what the highest-voted post in GamerGate’s subreddit was the day the Supreme Court ruled on the constitutionality of Gay Marriage?
It was this:
That doesn’t even have anything to do with gaming and it still hit the top of the subreddit (though, I suppose “Gay Marriage” kind of sounds like “Game Marriage”). We’re really going to lump that group in with “Conservatives?”
“But you’re undervaluing the existence of that 16.5% Pro-Life group” I pretend you say — because it helps me segue to my next point.
No. I’m not. That 16.5% Pro-Life portion is just as much a part of GamerGate as the 80.3% Pro-Choice portion. They aren’t going to split because they find each other’s ideologies icky. If you want an identifiable constant among GamerGate members it’s that they are just absolutely not playing a game of political cooties anymore.
The tweet that launched this study gave rise to some pretty lively debate. Opinions ranged from polarizing to nuanced to darkly comedic.
And, as is inevitable for topics like this, some conversations got heated. But ultimately everyone came away on good terms.
I struggle to imagine the mindset that allows people to find this sort of thing anything but endearing — but in any case, GamerGate isn’t conservative. It isn’t even apolitical. It’s just tired of your bullshit.
