So this “Receive or Pull?” analysis is great and all, but do we have any sense of how it relates to real game data? Let’s take a look at offensive hold probabilities at WUGC 2016 and in the AUDL.
Empirical Data + Win Probability
I considered looking at the percentage of games that a team wins and whether or not they start the game on offense. I worry that this calculation will be misleading as it’s difficult to interpret the wind effects for all the games in the data set. For instance, less than 50% of teams starting on offense and winning games would be consistent with this analysis as long as they are the teams starting upwind each game. Also, we would need to control for the strength of teams. Because of the difficulty controlling for these factors, I chose not to conduct this calculation.
WUGC 2016
The WUGC 2016 website has point by point data for all 121 Men’s games and 105 Women’s games from the tournament. This includes 2,769 Open points and 2,323 Women’s points. These data were scraped from the WUGC website for analysis.
From these data, I want to explore three assumptions. (1) What is the probability for each team to score offensive points? (2) Is there any correlation between a team’s offensive probability to score a point and if they were broken the point before? and (3) Do offensive hold rates vary for the first point in games or as the game progresses? It should be noted that all the data come from one tournament. Ideally the data would come from a variety of tournaments, but these data are easily accessible so let’s see what we got.
- (1) What is the probability for each team to score on each other?
A simple analysis would be to just measure how many offensive points were scored compared to breaks in the tournament.
Now this number is not truly indicative of teams’ offensive probabilities because worse teams are on offensive more often. Think about the game’s offensive hold rate for Team USA’s Men’s 15–2 win against Team Mexico. The offensive hold rate in this game is very low.
To determine each team’s offensive probability against each opponent, a logistic regression was run. From this, the succeeding plots show the average expected offensive probability for each team against every other team at the tournament. This is Model #1 in the tables shown later in this article.
The average expected offensive hold probability for all teams was 65.0% for Men’s and 64.0% for Women’s. My hunch is that when thinking about USAU club games the percentage of offensive holds is probably higher as the caliber of teams is better on the whole compared to the WUGC field.
- (2) Is there any correlation between a team’s offensive probability to score a point and if they were broken the point before?
I was really interested in seeing if there is any statistical significance to a team’s probability of holding on offense and if they were broken the previous point. My initial thought is that at the highest levels of the game, the teams are deep enough and conditioned enough that a break on the previous point should not affect a team’s chances the succeeding point. This can be nitpicked for tournaments with a strong upwind and downwind end zones.
From the regressions run for the WUGC 2016 tournament, there was statistical significance to including a variable for if the team was broken the point before for the Men’s division but not for the Women’s division (Model #2). Keep in mind that the regression controls for which team is on offense and which team is on defense each point.
A flaw with the regression though is that the number of observations of an offensive point after a break is higher for worse teams, so take this into account when assessing the magnitude of this correlation. For example, there are 64 points in the data set of the Egyptian Men’s team playing a point after they were broken, and there are only 7 of these points for the USA Men’s team. It would be great to have a much larger data set to really dig into this effect for each team as opposed to one effect correlated with all teams.
Here is a table that shows the average expected offensive probability for a hold for all teams in each division if the point was played after a break or not.
- (3) Is there any change in offensive hold rates for the first point of the game? What about any change in offensive hold rates as the game progresses?
There is no statistical difference between models that include these as variables compared to ones that just include the offensive and defensive teams in each point. From this tournament, the data show that there is no impact on a team’s offensive chances if it’s the first point in the game (Model #4) or as the game progresses (Model #5).
AUDL (2014–2016)
I analyzed raw data on ultianalytics.com for all the AUDL games from 2014–2016. I have to say that the raw data is a bit messy, so these results are not precise. There appear to be a lot of input errors in the data. Regardless, for looking at trends, these are good data to analyze. Note that I removed all points that ended with the clock expiring and no goal scored. This gives a more accurate predictor of offensive holds versus defensive breaks.
- (1) What is the probability for each team to score on each other?
Here are the data of offensive holds league wide for each season. Again, keep in mind that worse teams are on offense more often. The offensive hold rate is increasing each year. This makes sense as the caliber of AUDL play has been increasing with expansion teams and more elite club players participating.
- (3) Is there any change in offensive hold rates for the first point of the game? What about any change in offensive hold rates as the game progresses?
AUDL rules for who receives at the start of each quarter provide a good experiment on whether teams are more likely to be broken the first point of each game. In every AUDL game, teams that receive to start the 1st quarter are the same that start the 3rd quarter and vice versa for the 2nd and 4th quarters. Here are the hold rates for the start of each quarter. Data are for all seasons 2014–2016.
There appears to be no significant difference in a team’s probability of scoring the first point of the game compared to the first point out of halftime. The same is true for the start of the 2nd quarter and 4th quarter. I would like to do a more rigorous statistical analysis of AUDL points, but the amount of data cleanup required is significant.
Other Data
There have been other studies done to calculate offensive hold probabilities in ultimate games. Here is a sampling of ones I’ve come across.
· MLU 2013–2015
From Luke Ryan at MLU, the first three MLU seasons had 4,464 goals with 3,044 offensive holds resulting in a hold rate of 68.2%.
· 2016 USAU Women Club Games
Dusty Rhodes had a blog post with statistics on 16 games from the 2016 USAU women’s club regular season. He calculated 267 offensive holds in 386 points resulting in a 69.2% hold rate. His data also revealed that hold rates were higher in the 1st halves (74.5%) compared to the 2nd halves (63.2%) for these 16 games.
· USAU Men’s Club Games 2013–2014
The following post on Reddit provides data from elite men’s club games from 2013–2014. The ~500 points included a hold rate of 74%.
· USAU Club Nationals Finals
To build some intuition on hold rates and games, we can look to the USAU club finals in 2016. Ironside defeated Revolver 14–13. Ironside converted 10 of 13 offensive points (76.9%) while Revolver converted 10 of 14 offensive points (71.4%). Brute Squad beat Riot 12–11 in the women’s final. Brute Squad converted 8 of 11 offensive points (72.7%) while Riot converted 8 of 12 offensive points (66.7%).
· What should you select during the flip?
Well what about winning the flip? Always pick “even” or “same.” Look here if you’re interested in reading more into it.
