Graphing Rotten Tomatoes’ Political Bias
When Dave Chappelle’s comedy special Sticks & Stones was released in September, 2019, it got the distinction of being one of the most universally panned comedy specials by professional critics, while simultaneously being one of the most highly-rated by ordinary viewers.
On RottenTomatoes.com, the special was given a mere 27% on the Tomatometer, which is an aggregate score of professional film critics. This doesn’t come as a surprise considering Chappelle covered many controversial topics — from the LGBT community to abortion, hate crimes, the opioid crisis and gun culture — and did not exactly tackle them delicately.
At the same time, however, 99% of user-submitted reviews were favorable. While you expect some discrepancy between the masses and the trained critic, the difference here was startling. Why would almost 3 out of 4 critics find him appalling, yet the audience almost universally find him funny? Are critics that out of touch with the common viewer? Are average viewers that insensitive? Or, was this Rotten Tomatoes page simply overrun by trolls?
To answer these questions, I decided to compile some data. Because Rotten Tomatoes has both critic scores and audience scores for every film, one could compare the two over a large sample of films. This would allow me to test whether or not professional critics appear to have a bias — more specifically, a favorable bias towards politically “progressive” films, and/or a negative bias towards “problematic” films. (I’ll define these more clearly later.)
To be clear, my intention is not to pass judgement on any of these films themselves. Rather, if there is, in fact, a significant critic bias, it’s of interest to me to understand why it exists and how it might be influencing today’s political zeitgeist.
Critic Bias For Progressive Films
To start, I selected 20 films at random that were considered “progressive” by critics. To find them, I looked at several different lists, including these from Complex, Vice, PRA. These were films that dealt with issues of race, women’s rights, gender/LGBT identity, or got praise for diverse casting. I kept it limited to just narrative feature films (no documentaries, TV shows, etc.) and only those released in the last 5 years.
Then, I recorded both the critic score and the audience score for each film, and subtracted them to find the difference. The formula went critic score - audience score = difference, so a positive result meant that critics rated the film higher than the audience, and a negative result meant that the audience rated the film higher than critics.
I found that, overwhelmingly, critics tended to review progressive films more favorably than the audience. Of the 20 films in the sample, only Tangerine had a lower critic score than it’s audience score, and that was by a margin of 1% (critics gave it 88% while viewers gave it 89%). Hidden Figures had an even score (both viewers and critics gave it 93%), and rest all had critic scores at least 10% higher than the audience.
At the extreme end of the chart, Touch Me Not, an abstract film that dealt with alternative sexualities, was rated 59% by critics, yet only 15% by the audience; a difference of 44%. Likewise, the films Revenge, a feminist action thriller, and Jeune Femme, a French drama, both received critic ratings over 30% higher than the audience.
Critic Bias For Problematic Films
Next, I looked at films that were considered “problematic” by critics. These are films that were accused of cultural appropriation, whitewashing, sexism, racism, homophobia, transphobia, or for celebrating toxic masculinity. To find these, I again looked at lists from critics, such as these from Slate, The Washington Post, or Wikipedia. Also, to make sure it wasn’t just one critic’s opinion, I only included films that had multiple accusations on their Rotten Tomatoes page.
As with the progressive films, I kept it limited to just narrative feature films (no documentaries, TV shows, etc.) and only those released in the last 5 years. This way, I could rule out the influence of reviews that were written, say, 10 or 15 years ago, during a different cultural climate.
The result was an almost perfect inverse of the “progressive” chart. 16 of the 20 films had a lower critic score than the audience score. At the extreme end was 2015’s Stonewall, which head a 10% rating from critics, yet 85% from audience members. Interestingly, Stonewall had a progressive theme (the 1969 Stonewall riots for gay rights), but was heavily criticized for its lack of diversity in casting. Next was Loqueesha, a newly released comedy that has been widely panned as racist, and The Ottoman Lieutenant, a historical film that was criticized for “whitewashing” (featuring a largely white cast, despite taking place in a non-western setting) and also received some accusations that the film perpetuated the denial of the Armenian genocide.
The four films that had a higher critic score than the audience were Isle of Dogs, a recent Wes Anderson film that was accused by some of cultural appropriation, Whiskey Tango Foxtrot, a 2016 film that stirred some mild controversy for having white actors portray Afghan characters, Death Note, a Netflix film that was heavily criticized for whitewashing, and Once Upon a Time in Hollywood, Quentin Tarantino’s recent film that’s taken some heat for lack of diversity.
Critic Bias: Control Group
To help interpret these results, I also looked at a control group of 50 completely random movies, taking 25 films from two different random movie generators: GeneratorMix and BestRandoms. Adding up all their scores on Rotten Tomatoes, I found the average critic rating to be 75%, and the average audience rating to be 78%. In other words, professional reviewers tended to be slightly more critical than ordinary viewers, but only by a margin of 3%.
From that list, I also took a random sample of 20 films and graphed them the same way as my other charts. The result is almost a perfect balance; 9 of the films were preferred by the audience, 10 of the films were preferred by the critics, and 1 film was preferred evenly by both.
What Does This Mean?
Compared to the general audience, professional critics tend to give better ratings to movies with socially progressive themes, and lower ratings to movies that have stirred some kind of politically incorrect controversy.
This is not the same as saying critics like socially progressive films and dislike problematic films. That would be fairly obvious, as a film that gets bad press is likely to get bad reviews. Rather, when compared to how the audience rates a film, critics tend to rate progressive films 21% higher, and problematic films 13% lower.
Why is this the case? Let’s dig into a couple theories:
1. Audience Members are Trolling
In the last couple years, alt-right trolls have led campaigns to trash movies they see as having an opposing political agenda, including Star Wars: The Last Jedi, Black Panther, Captain Marvel and 2016’s Ghostbusters. When I checked these Rotten Tomatoes’ pages, I did see a lot of audience reviews from accounts with only 1 review. For this reason, I decided to exclude these four films from the sample.
For the rest of the films, however, I didn’t find much evidence of trolling. For one, most reviews seemed to be coming from genuine accounts (accounts that have been open for a while and reviewed multiple films). Secondly, many of these films were lesser-known indie productions that probably wouldn’t have been on the alt-right’s radar.
Furthermore, even if trolling is partly to blame, it does not explain the reverse trend seen for problematic films. I did not find any evidence of trolls “voting up” any of the problematic films for the sake of making their ratings seem higher (they did do that for 2012’s Hating Breitbart, but that’s a whole other story). Not to mention, several of the problematic films had progressive themes (Stonewall, Nina, Green Book) and are therefore pretty unlikely to be embraced by the alt-right.
2. Journalism Today Requires Extreme Opinions
In an effort to capture the attention of Twitter surfers and gain those precious ad views, it’s possible that critics are getting more extreme with their praise or admonishment. A headline with bold, tantalizing adjectives is more likely to get clicked on than a nuanced one.
While this makes for an interesting theory, it would be hard to prove without a significant research project — say, comparing recent film reviews to those 15 years ago and seeing if there’s a difference in language and ratings. I wasn’t able to find any data on this… but perhaps another day.
3. Professional Critics Lean Left
Maybe we’re looking too far into it. Professional critics tend to have college degrees, and the highly educated are more likely to lean left. Not to mention the film industry is located in and around Los Angeles, a heavily blue region.
While this can likely account for some of the bias, most critics would scoff at the idea that their rating is simply a manifestation of their political leaning, and not a careful analysis of the film’s fundamentals. Not to mention, Hollywood has leaned left for generations, yet the extreme polarization seems to be a more recent phenomenon.
4. Critics are Pressured to Conform
While I do think all of the above theories could help explain some of the critic-audience gap, commentators have proposed another idea that seems to get more at the heart of the phenomenon.
Owen Gleiberman in Variety wrote how the increasing importance placed on Rotten Tomatoes scores by moviegoers has put a subtle pressure on critics. “If you’re out of step with the majority,” he writes, “then you’re now in the position of not helping the cause… And your off-kilter opinion is going to be spotlighted — almost put on trial — by Rotten Tomatoes.”
Indeed, the layout of Rotten Tomatoes makes a negative review on an otherwise lauded movie stick out like a sore thumb (and vice versa).
While it’s not clear exactly how widespread this is, there have been instances of reviewers getting harassed for diverging from the dominant opinion. For example, women who reviewed 2019’s Captain Marvel negatively were called out on the left-wing twittersphere, as Washington Post reported.
But moreover, Gleiberman believes its a sort-of self censorship, brought on by critics who feverishly check what others are saying before writing their own review, “snorting up their opinions like lines of cocaine” and “ingesting their collective wisdom.” It’s not hard to conceive how having everybody’s opinion available at your fingertips — and in real time, no less — could help create a culture of conformity.
Why It Matters
At this point in the article, you’re probably thinking one of two things. If you lean right, you’re probably thinking BINGO! Of course movie reviewers are biased, just like the rest of the media. If you lean left you’re probably thinking so what? Who cares if reviewers prop up films with positive messages, and downplay films with hurtful ones — this is good for society.
The fact that so many people are going to walk away with wildly divergent opinions is precisely why conformity is so harmful. Critics may think they’re endorsing a good cause, introducing narratives into the mainstream that would’ve been marginalized 10 or 15 years ago. But they’re really just alienating a significant portion of the population — and precisely the voter blocks that liberals would most like to reach.
It’s not that the general audience doesn’t want to see films that deal with gender identity and race. It’s that they can tell when a rating is artificially inflated; when the plot isn’t all the way there; when it’s more preachy than it is narratively evocative.
Responding to criticism for casting a white male lead, Stonewall director Roland Emmerich told Buzzfeed “I didn’t make this movie only for gay people, I made it also for straight people.” Unfortunately for him, such heresy resulted in a 10% critic rating, effectively burying the film from a mainstream audience. Would a middle-of-the-road LGBT blockbuster have been so devastating for the general public? Unfortunately, we’ll never know.
