Using Data Science to Investigate NBA Referee Myths (NBA L2 Minute Report)
Do NBA referees really “swallow the whistle” ? Does star treatment exist in the last two minutes? Which referees are good and bad? I answer all of these questions using advanced data visualizations.
How many times have you sat on your couch, watching your favorite NBA team, and yelled at the refs for not calling a foul? Or nearly thrown your remote at your TV because James Harden is at the free throw line again (ugh). I know I’ve turned to my buddy and said “these referees are totally getting paid off” at least 100 times in my life.
Well, now we can try to prove it.
Since March 2015, the NBA has been releasing its “Last Two Minute Report” which, according to NBA.com, is “a play-by-play report regarding all calls and material non-calls in the last two minutes of the fourth quarter or the last two minutes of any overtime period (if applicable) for all games (including playoffs) that meet designated criteria…Effective with the 2017–18 season, an L2M will be done for any game in which one team’s lead over the other is three points or fewer at any point during the last two minutes of the fourth quarter or overtime.”
Essentially, it’s a way for the league to be more transparent with teams and fans about the performance of the referees, while also holding the referees accountable for their actions. Total win-win.
But numbers never lie, so this data should be able to finally give us definitive answers to many long-time theories/myths about referee bias in the NBA. As a data scientist, I decided to crunch the numbers and give a final verdict on three of the most common myths, as well as breakdown which referees are “good” and which are “bad”. Let’s get started.
“Star Players Get More Favorable Calls”
This might be the oldest myth in the proverbial book of basketball (not the Bill Simmons classic that I read on the bus to school way back in 5th grade). Star players get more calls because they’re stars. It’s sorta the NBA’s version of seniority.
Or is it?
To investigate this theory, I created a statistic named “favorable calls”. A favorable call for a player is either a missed call on that player or a wrong call against that player’s opponent.
I then created four different scatter plots: one using Box Plus/Minus as my player talent metric, one using Player Efficiency Rating, one using Win Shares/48 Minutes, and one using Value over Replacement Player. Each scatter plot’s “response” variable, or y-axis variable, is net favorable call rate, which is (favorable calls + unfavorable calls) / the total number of reviewed plays.
While each scatter plot had a positive correlation between how good a player is and how many favorable calls they get per game, the “coefficient of determination”, or R² value, was very low, and therefore not representative of a strong correlation (bummer, I know). The highest R² value for any of the four graphs was the win shares/ 48 minutes scatter plot, which had an R² value of 0.065 — meaning that only 6.5% of the variation in Net Favorable Call Rate is explanable by the change in player talent level. So generally, better players don’t get significantly better treatment from the Refs. Yeah, James Harden — you’re off the hook.
VERDICT: FALSE
“Swallowing the Whistle”
“Swallowing the Whistle” is a common NBA theory that says that referees cease to make neccesary calls in order to avoid playing too large of a role in deciding the outcome of a game. Referees would rather a player makes a game-wining shot than he makes two game-winning free throws after a shooting foul because of the potential media backlash the referee could face if the NBA Last Two Minute Report reveals that the foul shouldn’t have been called.
Here’s an example of a game from January 2020 in which Chicago Bulls guard Zach Lavine is upset because he believes the referees failed to call a foul on his game-winning shot attempt.
In order to figure out if this theory is true, I created a histogram showing the number of incorrect non-calls by five second intervals during the last two minutes of close games.
As you can see, there is a clear upwards trend of missed non-calls as the game gets closer and closer to the end, with noticeable spikes around 60 seconds left (one minute remaining) and 24 seconds left (one shot clock/ full possesion left). I also color-coded the graph by the four most common types of incorrect non-calls (personal fouls, shooting fouls, offensive fouls, traveling) as well as “other” calls, and the graph shows that both personal fouls and shooting fouls increase as time winds down.
BUT STOP.
That graph is actually very misleading. Volume statistics like total incorrect non calls don’t always tell the full story because they’re not standardized for comparison — in this case, there are more total calls as time winds down, and so we need to standardize for proportions.
Using incorrect non-call percentage as our new response variable, it’s now easy to see that swallowing the whistle does not, in fact, occur — there is no logical relationship between time remaining and incorrect noncall percentage in the NBA, and therefore we can assume that the difference in values is just random noise.
VERDICT: FALSE
“The Refs Favor Some Teams at Home”
Typically, home court advantage in basketball refers to the idea that fan support, easier travel schedule/ personal life accomodations, and more home games in a playoff series help the home team win. But what about the referees? Do they favor the home team?
As a collective, home teams recieved 1098 favorable incorrect calls versus 1068 favorable calls recieved by away teams. So the easy answer is, well, no.
However, I wanted to dig deeper. So, in general, home teams don’t have an advantage over road teams. But what about specific home teams? Do better teams with more stars, or teams that play in a bigger market and therefore are worth more money to the NBA, get more favorable treatment from referees?
To investigate this, I created a scatterplot of all qualifying NBA teams showing the relationship between favorable calls per game by average final point differential at home during “close games”. Essentially, the y-axis is showing potetial referee bias, and the x-axis is showing how clutch teams are.
This bar graph shows every single NBA team ranked from first to last in net favorable call percentage while playing at home. As you can see, the NBA’s most valuable franchise, the New York Knicks, check in somewhere in the middle, as do the Memphis Grizzlies, the NBA’s least valuable franchise. The difference between dead last and first is only 6.9%.
Similar to the swallowing the whistle graph, this 6.9% range along with the lack of any statistically signficant outliers proves that there is not a statistically significant increase or decrease in net favorable call percentage for any team, and therefore, no team receives any significant favoritism from the zebras.
VERDICT: FALSE
So…which referees are “good” and which referees are “bad”?
The last logical debate to breakdown is, among all the referees in the NBA, which ones perform the best in the clutch? The fairest way to judge a referee’s performance is to look at how many incorrect calls during the last two minutes of close games they make, and the results actually might suprise you.
Steve Anderson had the highest number of incorrect calls/ game at 0.114, while Mike Callahan had the lowest at 0.012, which means the range of values for number of incorrect calls/ game for referees with at least 20 games refereed is just 0.102 — essentially one extra missed call every 10 games. That’s such a shockingly small and insignificant number — and when you consider that Anderson’s 1.09 incorrect non-calls per game is among the best marks in this entire group, you really can’t consider him a “bad” referee. In fact, the only referee in the top-10 of incorrect calls/game that is also in the top-5 of incorect non-calls/game is Derrick Stafford, and even his totals aren’t substantially “high”.
VERDICT: They’re all pretty good.
There you have it — the referees really are not biased. They’re not biased towards star players, bad players, certain teams at home, or really anybody. As far as the “Swallowing the Whistle” thing, well, that’s probably not going to change — and because it is such an obvious, league-wide trend, it is definitely a conscious and pre-determined strategy by referees and therefore not an example of poor referee performance.
So no, referees don’t hate your favorite team, and therefore you shouldn’t hate them either (but who am I kidding, I know you still will).
The NBA Last Two Minute Report dataset used in this article can be downloaded here.
Follow Andrew Fenichel on Twitter: @Andrew_Fenichel
Connect with Andrew Fenichel on LinkedIn: Andrew Fenichel
