Scriptbook’s artificial intelligent gender measure confirms gender inequality in film
Gender disparities in the film industry are under heavy debate. Male presence dominates in filmed entertainment. Studies [1–4] show that, on average, there are twice as many male as female characters in films, and similar ratios can be found when looking at the gender of the main protagonist, the number of speaking lines per gender, or even the gender split of directors.
Efforts to develop and identify objective metrics to better understand gender bias is an area where artificial intelligence can be used. Recently, Google shared the article “The women missing from the silver screen and the technology used to find them”[5]. The 100 most grossing (US domestic) live-action films from 2014–2016 were screened with a machine learning tool called the Geena Davis Inclusion Quotient (GD-IQ) [6]. This tool focused on female-on screen time and female speaking time, and additionally recognized patterns the casual movie viewer might overlook.
The results uncovered unconscious bias and confirmed the gender inequality and underrepresentation of women based on on-screen analyses.
At ScriptBook, we developed an automated tool to measure gender bias by using deep learning to analyze screenplays. We consider a range of metrics such as the amount of dialogue and overall presence in the script devoted to male or female characters, and the amount of interactions between them. Furthermore we determine to what extent male and female characters within a script conform to gender stereotypes, based on their actions. The final test we perform is the so-called Bechdel Test. In order to pass the Bechdel Test, a movie must meet 3 requirements:
- The movie must contain at least 2 (named) female characters
- Having a conversation with each other
- About something other than a man.
Manually checking the Bechdel Test conditions can be a tedious task since one must check every single conversation in the movie for a possible pass. ScriptBook’s system is trained to recognize the gender of each character and decide whether a conversation is about men or not. When applying the Bechdel test and additional parameters to measure gender bias, we confirm the earlier findings of widespread underrepresentation of women in Hollywood, based on the sole basis of a script.
Bechdel test: results
First, we analysed how the thousands of scripts in our our database performed on the Bechdel test. Not surprising, only 38,5% of the scripts pass (see figure below).
By selecting scripts according to their genre, the results indicate that certain genres like romance and horror tend to score much better on the Bechdel Test than adventure, crime or animation movies. The results based on our automated script analysis confirm Google’s on-screen GD-IQ results for genre.
Another interesting analysis is how the Bechdel Test results vary through time. The figure below shows the variation when a timeframe is defined. For example, around 1980 only 25% of the movies passed the Bechdel Test. This number has increased throughout the years to 43% in 2017.
The Bechdel test and beyond
As previously stated, technology efforts are being undertaken to objectify and tackle the issue of gender inequality in the film industry. At Scriptbook we believe that the Bechdel test is an iconic metric, but designed more to make a statement than to be used as an ultimate litmus test. Therefore we developed a number of additional metrics that provide further insights in gender disparities in a script such as: the number of inter- and intra-gender interactions (male-male, female-female, male-female); a ‘stereotypicality’ measure that indicates whether actions performed by each gender are in line with stereotypical behaviour; the number of male/female characters in a script; the number of spoken lines per gender.
To illustrate The Bechdel test and the additional metrics, we present 2 use cases:
Use case — Mudbound (2017)
Mudbound is a critically acclaimed movie set in rural Mississippi, that tells the story of two intertwined families and the impact that the Second World War has on them. This movie passes the Bechdel Test, but only for 2 scenes.
As you can see on the figure above, our application shows the scenes and the dialogue that make the script pass the test. In this case it shows a conversation where Isabelle is playing in the mud when suddenly her pregnant mother, Laura, has cramps in her stomach. Laura then instructs her daughter to get help.
Even though this movie passes the Bechdel Test, our additional metrics show that almost 75% of speaking lines are from male characters and the male presence reaches 71.5%.
Interactions however seem a little more balanced since about half of the interactions involve a woman. This makes us believe that women are part of the interactions, but it is often men who talk or act.
The ‘stereotypicality’ metric further proves this point, Mudbound is among the most stereotypical scripts. This shows that passing the Bechdel Test is not sufficient to conclude gender equality.
Use case — Still Alice (2014)
Still Alice is a movie about a female linguistics professor who is diagnosed with Alzheimer’s disease. As indicated on the figure below, the film passes the Bechdel Test for multiple scenes.
This movie has a strong female lead and various influential female characters. This is also represented by our additional metrics: it has more female than male characters, a female presence of 74%, almost all interactions involve females and the female characters are also the ones who talk the most.
Perhaps the most interesting metric is the stereotypicality. As mentioned before, actions that indicate control, intent or decision-making are usually attributed to male characters. This movie however gives the female characters exactly that type of control. Therefore, Still Alice is marked as a highly non-stereotypical movie.
Awareness has impact
Artificial intelligence has proven to be incredibly powerful. It provides a fully objective view on subjects where we might have developed an unconscious bias. By exposing ourselves to objective metrics, we become aware of gender bias and are able to address them.
Identifying gender bias using artificial intelligence is just one area of possibilities. AI can give insight into much richer concepts such as personalities, character traits and emotions of characters and how they evolve throughout a story.
Implementing artificial intelligence to reduce unconscious bias such as gender bias will hopefully lead to a much fairer distribution of roles to women in filmed entertainment.
REFERENCES
[1] Bleakley, A.; Jamieson, P. E.; Romer, D. (2012). “Trends of Sexual and Violent Content by Gender in Top-Grossing U.S. Films, 1950–2006”. Journal of Adolescent Health. 51 (1): 73–79. doi:10.1016/j.jadohealth.2012.02.006. PMID 22727080.
[2] Sakoui, Anousha; Magnusson, Niklas (22 September 2014). ‘Hunger Games’ success masks stubborn gender gap in Hollywood”. Chicago Tribune. Retrieved 22 September 2014. With reference to: Smith, Stacy L.; Pieper, Katherine. “Gender Bias Without Borders: An Investigation of Female Characters in Popular Films Across 11 Countries”. See Jane. Retrieved 16 April 2016.
[3] Smith, Stacy L.; Choueiti, Marc; Pieper, Katherine; Gillig, Traci; Lee, Carmen; Dylan, DeLuca. “Inequality in 700 Popular Films: Examining Portrayals of Gender, Race, & LGBT Status from 2007 to 2014”. USC Annenberg School for Communication and Journalism. Retrieved 6 August 2015.
[4] Swanson, Ana (12 April 2016). “The problem with almost all movies”. The Washington Post. With reference to: Anderson, Hanah; Daniels, Matt. “The Largest Analysis of Film Dialogue by Gender, Ever”. Polygraph.
[5] The women missing from the silver screen and the technology used to find them. https://www.google.com/about/main/gender-equality-films/index.html
[6] https://seejane.org/research-informs-empowers/data/
[10] https://www.hollywoodreporter.com/news/bechdel-test-2016-movies-passed-failed-952944
