COGNITIVE HOLLYWOOD, Part 1: Data Shows Box Office Economics in Turmoil
NOTE: the opinion below is in my name only, and does not represent the views of ETC, the School of Cinematic Arts, or USC as a whole. Also, this analysis does not factor in It’s recent box office successes.
As part of my work managing the Data & Analytics Project at USC’s Entertainment Technology Center, I spend a lot of time analyzing data around the relationship between film stories and film audiences. Because of this, I believe the entertainment industry is entering a crisis, and much of this crisis has to do with the fact that audiences now demand increasing levels of innovation in how film and TV stories are told. They want novelty, and to satisfy them, Hollywood will need to profoundly transform how it does business. Filmmakers will need to make more data-driven decisions, and think more cognitively about their audiences.
Box Office Hurricanes As Evidence of a Change in the Entertainment Climate
Hollywood CFOs are closing out the Summer, and it’s a Lannister kind of party. The worst Labor Day Weekend performance in 17 years; a bloodbath Summer box office in a near 20% retreat compared to 2016; bulletproof franchises underwater; even the mighty Dwayne “the Rock” Johnson took a beating. Summer 2017 was an Irwin Allen disaster.
It’s not that the industry is having a bad year overall — so far the median worldwide box office revenue for 2017 is around $66 million, up from $61 million in 2016, and there are still a few months left to go.
The problem goes deeper than that.
Audiences seem to be turning away from some movies. The hit rate hasn’t yet come down too much, but there is ample evidence that, as production budgets (and financial risks) go up, volatility in financial performance is also going up. What this means is that the Traditional Hollywood Formula of Hit-Making is coming to an end, and studio executives aren’t ready for it.
Nowhere was this more clear than in last week’s article in The New York Times by Brooks Barnes, “Attacked by Rotten Tomatoes”, which echoed a view often heard among senior creative executives in Hollywood that poor Rotten Tomatoes reviews are to blame for bad theatrical performance. One anonymous executive even went so far as to put the destruction of Rotten Tomatoes (which is owned by Fandango, itself co-owned by NBCUniversal and Warner Brothers) at the very top of his goals for next year.
Debunking the Rotten Tomatoes Myth, With Data
Barnes’s piece was pretty high on posturing and pretty low on actual data. So, being the resident data scientist at the ETC, I decided to see what the math said.
I collected box office return data through Box Office Mojo for all the 150 titles released in 2017 that grossed more than $1 million, plugged in Rotten Tomatoes Scores and Audience Scores for all titles, and looked at the correlation between scores and financial performance through both a basic Pearson Product-Moment Correlation Coefficient (PMCC) analysis and some linear modeling to extract r-squares, a measurement of the strength of correlation.
PMCC measures the linear correlation between two variables X and Y. It has a value between +1 (100% positive correlation) and -1 (100% negative correlation, often called “inverse correlation”). The closer to 0 a PMCC score is, the less correlation exists between X and Y.
So has Rotten Tomatoes been spoiling box office performance? According to the data: Nope. The math shows pretty overwhelmingly that there was no correlation (positive or negative) in 2017 between Rotten Tomatoes Scores and box office returns.
The results showed a 12% PMCC correlation, and a .009 r-square, meaning there is likely no statistical relationship between the two variables. Even more surprising, the impact of Rotten Tomatoes scores on opening weekend box office seemed even lower: .08 PMCC score (a mere 8% correlation), and a -0.001 r-square.
That’s for all 2017 titles so far. What about the Summer titles, which the executives quoted by The New York Times complained about?
Again, nada. We found no meaningful correlation between Rotten Tomatoes scores and the total gross for summer movies running between May and Labor Day. We saw a .07 PMCC score (only 7% correlation), with a -.006 r-square, which is actually less correlation than for the entire 2017 slate.
There was even less correlation between Rotten Tomatoes Scores and opening weekend performance: -.03 PMCC score (meaning no relationship whatsoever), with an r-square of -.01.
It’s unclear how much of some creative executives’ opinions related in the New York Times article reflect actual belief that critics are hurting the top line, and how much they reflect the need to scapegoat Rotten Tomatoes.
What is clear, looking at all film data since 2000, is that Rotten Tomatoes scores have never played a very big role in driving box office performance, either positively or negatively.
Overall, Rotten Tomatoes scores for all movies grossing more than $2 million have been pretty stable since 2000. The median score was 51 during the 2000s and 53 during the 2010s so far, and it’s actually gone up quite significantly from 2015 (46.5) to 2017 (71).
For movies grossing more than $300 million worldwide, the median Rotten Tomatoes Score was 73 in 2013, then about 72 for 2014, 2015 and 2016. However, it has gone up to 77.5 in 2017, which means that — contrary to the grumble coming from Hollywood — critics are actually enjoying the slate this year.
Does a Rotten Tide Raise All Boats?
So Rotten Tomatoes Scores are going up, together with the median box office (adjusted for inflation), which has increased, from $26.6 million in 2015 to $66.1 million in 2017. But does the fact that Rotten Tomatoes scores and financial performance have both gone up suggest a relationship between the two variables?
Let’s — you guessed it — look at the data. Once again, using the PMCC as a measure of the strength of the correlation, and analyzing all films having grossed more than $2 million worldwide, we see the following:
This is actually a pretty strong correlation, which seems to be increasing, and — naturally — seems stronger with Domestic Box Office than Foreign Box Office performance (It’s unlikely that foreign markets, which are now much more profitable to Hollywood than the domestic market, pay much attention to Rotten Tomatoes scores).
What about movies grossing more than $300 million worldwide?
The correlation seems equally strong. Clearly there’s something going on here: could the executives who complained to The New York Times be right in accusing Rotten Tomatoes of destroying their industry?
Well, not exactly.
There’s another, very interesting phenomenon at play, one that holds deep implications for how Hollywood makes movies:
Audience scores and critic scores are increasingly correlated, meaning that audiences are becoming expert at smelling a “bad” movie and staying away.
Indeed, the data shows a very strong correlation between Rotten Tomatoes scores and audience scores (also reported on the Rotten Tomatoes website) for all films (that grossed more than $2 million worldwide) released since 2013:
This chart sums up the correlations:
Whoa. There’s virtually no difference between critics’ scores and audiences’ scores, and the more successful the film is at the box office, the smaller the difference. This makes it very difficult to isolate the true impact of Rotten Tomatoes scores on box office performance. When Hollywood executives complain about Rotten Tomatoes scores, they’re really complaining about their audience’s tastes, because it’s basically the same thing.
This suggests something that I’ve been thinking about for a few years, which is that it’s frankly a miracle that the entertainment industry would be making any money.
Imagine running a company in a shrinking market where every product costs hundreds of millions of dollars to make, where every single one of your customers is an expert in the product, and millions of competitors are offering free, lower-quality but very accessible and compelling knockoffs.
And that’s not even the worst of it. The worst part is that, as a studio executive, you know next to nothing about who these customers are, what they want, or why they like your product. That’s Hollywood.
The market for entertainment has been shrinking as the amount of time we spend on entertainment shrinks (at least until autonomous vehicles roll around). See below (thanks to UCSD’s Jim Short for the data):
And no matter how old you are, you’re a film expert. How many hours of film and TV have you consumed since birth? Compare that to how many cars you have bought in your lifetime.
Meanwhile, the competition for this shrinking pie (digital and social media) is free, or near-free, and it’s exploding. Think about how many options you have to entertain yourself right now, between linear TV and film, OTT, social media, streaming music, the web…
Yes, the fact that the industry not only still exists, but is turning healthy profits overall, is a testament to the genius of those who run it (and the insanity of those who fund it).
RIP: the Good Old Days
Let’s make this as clear as possible: no Hollywood executive knows why you liked War of the Planet of the Apes, or why Game of Thrones is one of the greatest successes in the history of entertainment. Creative decisions across the entire industry are still largely made on the intuition of a handful (no more than 50) of executives who rely less on data and more on the cognitive models developed over decades of bone-crushing experience making movies.
These executives were bred over years and years of working the system from the inside. From the mailroom to the assistant’s cubicle to the VP of Production desk, they were bathed in the infinite human complexities of conceiving, creating and selling entertainment, slowly establishing their own cognitive scaffolding (i.e. biases) for what makes the ‘magic of story’ work. Their brains physically experienced what neuroscientists call “neurogenesis”: a profound restructuring of the brain that can lead to a change in how one perceives the world.
By the time they would rise to the top of the studio system, executives had become film. They had become TV. Success was defined by the ability to bring together the basic element of success: creative talent. Bring it, pay it, and they will come.
Nobody ever got promoted in Hollywood because they had better inside knowledge of what audiences wanted to see, simply because, until fairly recently, that didn’t matter. The industry had an unassailable stronghold on our leisure time, and so the factors of success were, for the most part, intrinsic (the product), rather than extrinsic (the market).
This is why, as CGI opened the door to a much more immersive form of storytelling, and the landscape became more and more crowded with smaller and more nimble competitors (Miramax, Lionsgate, etc.), median production budgets surged: from $8 million (inflation adjusted) in the 1970s, to $22 million in the 2010s.
And now with even more competition from such newcomers as Netflix and Amazon (and soon Facebook) that median is rising fast: $25 million in 2016 and $30 million in 2017.
Lately the rationale has been: make more expensive films (heavy on extremely expensive visual effects and CGI), and you’ll attract larger audiences, especially in foreign markets.
Is that true? Let’s (say it together now!) look at the data:
This is pretty compelling, and seems to validate the traditional Hollywood view of higher production budgets as a means of ensuring higher returns. Interestingly, we clearly see here that the introduction of CGI, which drove production budgets up, created value for audiences in the form of more spectacular and experiential content.
This is basic economics: if our time to entertain ourselves is flat (flat demand) but the number of options we have to entertain ourselves for the same amount of time is exploding (rising supply), then rational actors (us) will maximize the cognitive payoff (here, the dopamine rush that entertainment produces) by investing our scarce resources into the most cognitively rewarding (eg the most experiential) forms of entertainment possible: big, CGI-laden blockbusters and video games.
For more than 20 years, Hollywood’s bet on inflated production budgets was a good one, especially as they conquered foreign markets, here there seemed to be a particular appetite for these type of films. Until now.
After 20 years of big and loud CGI titles, there’s substantial evidence that audiences are tuning out. As CGI reaches the upper limits of photo-realistic characters and scenes, audiences are getting bored. Action-driven blockbusters high on visual effects but low on finely-crafted characters (which actually worked better to sell a film across cultures) are starting to lose money at the domestic box office with increasing frequency.
And it’s starting to show in the data:
Since 2013, the correlation between production budgets and domestic theatrical performance has started to come down, and it’s even looking soft on foreign markets.
What about for big box office earners?
Here it seems the story is a bit more complex, but at a minimum, the correlation seems to soften, and the volatility in PMCC scores suggests that box office performance is driven by variables other than production budget, or that need to be combined with production budget to generate financial performance.
Let’s see if we can collect some insights by turning to another dimension in the relationship between audiences and film. For 20 years, rising production budgets led to the production of films that felt more and more enjoyable to audiences.
Clearly the increase in CGI budgets didn’t impress the critics much. But while the impact was rather small on audiences, we see a substantial rise in how much audiences enjoyed movies during the Golden Age of CGI, from the 2000s until now.
Is this still true? Let’s turn to the data once again:
Let’s look at this another way: is there a relationship between critics and audience scores and production budgets? See below.
Three things are apparent here:
- The New York Times article notwithstanding, Hollywood is having a great year with critics and audience scores so far;
- There’s definitely a lot of volatility in the scores, regardless of the production budget, meaning there are hidden variables (beyond budget) influencing scores;
- It’s very clear that production budgets do have a very substantial impact on both critics and audience scores. Here again it seems Hollywood’s bet on CGI has largely paid off.
However, the data above also suggests three more profound (and challenging) trends for Hollywood:
- Production budgets are less and less of a predictor of box office success, meaning that as financial exposure rises, so does financial risk. This is not good (for a long time it was the opposite), and is a substantial reason why Wall Street has been so tough on entertainment stocks lately;
- Audiences are becoming extremely adept at predicting and judging the quality of a film;
- Quality here is a very subtle measure of how much, and where, a film innovates in its story and character mechanics, and how much — and where — it doesn’t.
Part two of this post will go into why — and how — we’re entering a new era in Hollywood, the Golden Age of Stories, and why the industry as a whole has no choice but to think cognitively about storytelling.
