A Statistical and Methodological Breakdown of The Video Games =/= Sexism Study #GamerGate

Hello everyone, I hope you’re well! I typically post GamerGate-related content to my Twitlonger, but I decided to migrate to Medium for this due to the ability to conveniently post images and links.

A newly-published article has been making the rounds on social media which refutes the claim that exposure to video games influences the development of sexist attitudes. The abstract for said study can be read here. I recommend reading it before continuing as I will be going over the study in-depth rather than discussing what is available to everyone.

I’ve decided to put on my researcher hat and break down the findings of the study for everyone to understand. I was originally only going to break down the statistics of the paper, but I will also break down the methodology of the study.

Just a quick note before I begin. This study was part of a national telephone survey in Germany which seems pretty big. It’s not uncommon for researchers to collect huge amounts of data, analyse them with different hypotheses and publish them separately (hell, I’m part of one right now). I recommend for people to watch this space, there may be future video game-related research coming out of this study.

Each section will have a subheading. Please navigate as you please!

The Methodology — The Bad.

• Participants didn’t have an awful lot of choice when recording video game genres. According to the appendix, participants were asked how often they played and how much they liked ‘Role-playing games’, ‘First-person shooter games’ and ‘Other action games’. Participants had the option to say that they didn’t play these types of games, but they had no option to volunteer their own categories. I find it a bit hypocritical that they single out fighting game women in their abstract, yet don’t even look at this genre. However, the researchers acknowledge this limitation and suggest it as an avenue for future research. I would also play Devil’s Advocate on their behalf and justify it on the basis that they looked at each category’s relationship with sexist beliefs. Things get a bit messy when participants start volunteering their own categories (“What do you MEAN you haven’t heard of Touhou? Are you some sort of casual?”), so I can understand them trying to control their categories for individual analysis.
• This is Criticism 101, but always beware of taking one population’s findings and applying them globally. What applies to Germany may not apply to Japan, Sweden etc.

The Methodology — The Good

• This is a pretty good sample. 824 participants (464 male, 360 female) over a three year time period in a naturalistic setting (telephone conversation at home). I do like that it’s relaxed and natural rather than bringing people into an artificial environment and asking them to answer questions on sexism.
• 902 participants were involved in all three stages of the study, but the researchers took no shit when managing their data and deleted anyone who missed out even one question (known as ‘listwise deletion’). This is arguably the best practice for handling missing data, but it can be scary as you can end up losing lots of participants. Credit where credit’s due for trying to maintain the best possible data.
• The sexism scale used is very reliable. I wanted to single this out because the researchers specified condensing the scale for ease in telephone interviews. This can be an absolute nightmare as removing questions or simply re-wording questions can make them very unreliable and inappropriate to use in research (I actually know this feel…). The fact that they shortened the scale and it retained high reliability is a good sign that they were measuring important aspects of sexism.

Now on to my favourite part, the statistics!

Structual Equation Modelling or: How I Learned to Stop Being Critical and Listen to Anita.

Apologies if the subheading is a little abrasive, it was genuinely my first thought when I read the results section of this article. Hopefully if I’m clear enough, the subheading will be clear to you soon.

Structural Equation Modelling (SEM) is one of my favourite types of statistical analysis. In layman’s terms, SEM tests a ‘model’ which you have developed based on theory and/or evidence. Let’s take the current study for example. The researchers hypothesized that sexist attitudes are related to video game use and can predict future video game use. While testing this, they also wanted to look at this relationship when age and education were factored in. Not only that, but they wanted to explore the relationship for sexist attitudes and video game use during the different time points. To massively condense that, the story would be “I think sexist attitudes relate to video game use, encourage video game use and that the two are consistent during time periods”.

Confused? Could you use a diagram? Thankfully, SEM is all about the diagrams!

This is the SEM diagram presented in the paper. Don’t worry about the lines and numbers for now, I will break these down soon. Instead, I want to focus on the ‘fit’ of the model. Model fit will tell us how accurate our theories are in predicting what we want to predict. For this case, SEM will tell us how good sexist attitudes are in explaining video game use and how influential video game use is to sexist attitudes. The fit criteria is presented below:

The section I have underlined is referred to as the ‘Chi-Square Fit Criteria’. This tells us whether our model can be significantly improved upon. A tiny, non-significant (p > .05) Chi-Square value says to us “Wow, you were right all along! Video game exposure has a huge impact on sexist attitudes and vice versa! Video games explain sexism so well that there is absolutely nothing more we can add to our story that will explain sexism! You’ve solved the mystery!”.

This is one of the largest, most significant Chi Square values I’ve ever seen. For reference, here is some from my research:

Hopefully my abrasive subheading now makes sense. SEM is telling us to get real. This model is ineffective at explaining sexism and video game consumption and massive improvements can be made to it by considering a wealth of other factors.

The video games = sexism argument is not statistically sound. The narrative is crumbling. The reductionism is bullshit.

As promised, I will now break down the arrows and numbers. I will also discuss the relationship between age and education that was left out of the diagram.

• Sexism attitudes remain consistent across time, as does video game consumption. However, the authors noted that there were gender differences — women were more consistent gamers than men. Pretty cool huh?
• Video game use was not related to sexist attitudes between any time points. However, there was one significant correlation across time points. Video game use for males lead to a significant decrease in sexist attitudes between two time points. However, the authors rightfully comment that while the relationship is significant, the change between timepoints is so small that they consider it ‘negligable’.
• There was no relationship between genre of game preferred, sexist attitudes or gender.
• Males hold less sexist attitudes as they get older, but this is unaffected by video game consumption. More educated females play more games than less educated, but this is not related to sexism.

Wow, this turned out to be longer than expected. I hope it was useful for at least some of you and I can always be contacted if anything did not make sense.

Lots of love and happy gaming.