Modelling Batter Decision Value

Introduction

Quantifying baseball swing decisions involves a comprehensive analysis of a batter’s approach against pitches and the locations which they are thrown. The most successful baseball players consistently swing at favourable pitches while laying off unfavourable ones. Understanding and quantifying these swing decisions form a crucial aspect of analyzing a batter’s performance.

Pitch Outcomes

There are many different outcomes that can occur during each pitching event throughout a baseball game. A high level overview of pitch outcomes would reduce the number of possible outcomes to five (5). They are the following:

1. Swinging Strikes (Including Foul Tips)
2. Called Strikes
3. Foul
4. Hit into play
5. Ball (Including hit by pitch)

For each pitching event, MLB provides the change in Run Expectancy before the Pitch and after the Pitch. This is referred to as “delta_run_exp” in their documentation. This metric provides information that can assist in quantifying the value of each swing decision a batter can make. Delta Run Expectancy seems like a great choice as the target in training a machine learning model.

“delta_run_exp” includes the change in run expectancy for each pitch, but in the game of baseball, not every pitch is created equal. Depending on many factors, such as count, runners on base, and outs, the “delta_run_exp” can differ drastically between two pitches with the same outcome. For example, a pitch is thrown to the batter in the following outcomes occur in the given scenario:

1. a swinging strike with no runners on base and on an 0–2 count with 0 outs
2. a swinging strike with the bases loaded on a 3–2 count with 2 outs

Despite both outcomes resulting in a swinging strike, these two outcomes result in the following average “delta_run_exp”:

1. -0.15
2. -0.94

This makes sense intuitively, as in scenario A, the batter was in a situation where they already lost most of their expected run value because they were down in the count. Additionally, with no runners on, the batter didn’t strand anyone and most of his lost value comes from simply getting out. In scenario B, the batter was in a situation in which they could have provided immense value to their team. With either a ball (inducing a walk) or a hit, the batting team is guaranteed to score at least 1 run. However, the batter struck out, leaving 3 runners on base and ending the inning in a prime scoring situation.

These drastic differences in “delta_run_exp” can pose an issue when being used as a target in a machine learning model because the run value is situation dependent. To account for this, the average “delta_run_exp” for each of the outcomes can replace the “delta_run_exp” during training.

Figure 1 shows the average Mean Delta Run Expectancy for each of the outcomes throughout the 2020–2022 MLB seasons.

Figure 1: Outcome Mean Delta Run Expectancy

An interesting observation pops up when comparing the average run value of Called Strikes to Swinging Strikes. Under normal circumstances, these outcomes are the only ones which can result in the exact same result: a strike, and if there are 2 strikes already, a strikeout. So why does the average run expectancy differ so much between these two outcomes?

Functionally, they cause the same result, but this discrepancy stems from the volume at which these outcomes occur. A strikeout causes the most negative expected run value. Taking a look at the proportion of each outcome results in a strikeout paints a clearer picture. Called strikes results in strikeouts 8% of the time, while swinging strikes skyrockets to 34.4 3rd Strike%. Taking the average does not capture an accurate representation of swinging strikes and called strikes. This can be applied further to say that the average most likely doesn’t represent any of the outcomes correctly.

Figure 2: Called Strike and Swinging Strike 3rd Strike Rates

If the average of these values are to be used as the target value in a machine learning model, the average should not only be taken on the outcome, but also consider the count as well. Figure 3 displays the expected run value of each of these outcomes, considering the count.

Figure 3: Pitch Outcome Mean Delta Run Expectancy by Count

Swinging and called strikes seem to result in the same run expectancy, given that each of those outcomes occurred in the same count. Understandably, a swinging strike or called strike with 2 strikes would result in the most negative run expectancy, as the plate appearance would (generally) end in an out in this situation.

Additionally, fouls on non-2 strike counts have a similar result as called strikes and swinging strikes in the same counts. This makes sense as they all result in a strike against the batter. In the same vein, fouls on 2 strikes result in no change in run expectancy, as they do not cause any changes in the situation.

The more balls there are in the count prior to the pitch being called a ball results in a greater run expectancy. This makes sense, as the more balls a there are in a count, the closer a batter is registering a base on balls.

A good question arises when looking at strikeouts and walks. Why does count impact the value of a walk and a strikeout? For example, striking out on a 3–2 count is approximately double the (negative) run expectancy as a strikeout on an 0–2 count. Similarly, a walk on a 3–2 count is approximately double the (positive) run expectancy as a strikeout on a 3–0 count. One explanation for this is how the other outcomes would impact run expectancy. For example, since a 3–2 count would result in a walk if the pitch was a called a ball, the swing in run expectancy would be larger than striking out on a count which wouldn’t result in a walk if a ball was called. The same logic holds for walks, where walking while at risk of striking out has as higher run expectancy than while not at risk of a strikeout.

Hitting into play is not as easily defined as the previously discussed pitch outcomes. For the average run expectancy for a hit into play, all the hits into play were categorized into the following groups:

1. Field Out (Including errors)
2. Single
3. Double
4. Triple
5. Home Run

Figure 4 illustrates the run value for each outcome given the specified count.

Figure 4: Hit Into Play Outcome Mean Delta Run Expectancy

Hits that do not result in outs do not have their mean delta run expectancy greatly impacted by count. The discrepancies that do arise (e.g. 3–0 Triples, which have occured just 2 times in 2020–22) can be attributed to a lack of sample size. For field outs, there is a similar pattern to how the value of walks and strikeouts are influenced by count. In counts favourable to the batter, hitting into an out results in a greater negative run expectancy.

For the purpose of modelling, it is assumed that the average of each of the outcomes by count are an accurate representation of their run expectancy when it comes to swing decisions.

Swing Outcomes

In the simplest terms, for any given pitch, there are two swing outcomes:

1. The batter swung at the pitch
2. The batter did not swing at the pitch

Figure 5: Outcome Tree

Deciding which pitches to swing can be seen as a function of pitch location, and as discussed previously, the count in the current at bat. Figure 6 illustrates the expected run value of taking or swinging a pitch given its locations during 3 MLB seasons, spanning from 2020–22. In both plots, blue indicates favourable for the batter, while red indicates unfavourable.

Figure 6: Mean Delta Run Expectancy Hexbin Plot

When a batter takes a pitch, there are 2 outcomes:

1. Ball
2. Called Strike

These outcomes are illustrated in the “Pitches Taken” plot in Figure 6. Pitches thrown inside the strike zone are generally called as strikes. As the pitch location moves towards the edge of the strike zone and beyond, the run expectancy value trends towards being favourable for the batter. In general, a batter’s in-zone judgment can be quantified by looking at the proportion of their takes. The smaller proportion of favourable “in-zone” pitches taken indicates that the batter is more likely to not swing at outside (blue) pitches than inside (red) pitches.

When a batter swings at a pitch, there are 3 outcomes:

1. Hit Into Play
2. Swinging Strike
3. Foul

These outcomes are illustrated in the “Pitches Swung At” plot in Figure 6. Pitches swung at inside the zone are more likely to return positive value than pitch swung at outside the zone. This can be explained by the location frequency of each of the outcomes, which is illustrated in Figure 7. Swinging strikes mostly occur in the upper and lower part of the zone, while foul balls and hits into play are mainly situated in the centre of the zone. As visualized in Figure 3, a hit into play is the only swing outcome that can result in positive run expectancy, so it is reasonable that locations with the most hits into play results in the most positive run expectancy.

Figure 7: Pitch Location Frequency of Swings

Now that swing the outcomes and their run expectancy have been defined, a machine learning model can be trained to predict the run expectancy of each pitch during the 2023 MLB Season.

Training The Model

Two XGBoosted decision tree was used to model the run expectancy of a pitch. The two models are:

1. Pitches which were taken
2. Pitches which were swung at

These swing outcomes were separated as they are very distinct outcomes and measure different skills. Model A (Pitches Taken) would quantify a batter’s “In-Zone Awareness” and Model B (Pitches Swung At) would quantify a batter’s “Out-of-Zone Awareness”.

Model A quantifies a batter’s “In-Zone Awareness” because a batter with a good awareness of pitches inside the zone would have a very large proportion of their takes as balls. This means that batter’s with good in-zone awareness are prolific at swinging at favourable pitches.

Model B quantifies a batter’s “Out-of-Zone Awareness” because a batter with a good awareness of pitches outside the zone would have a very large proportion of their swings at pitches inside the zone. This means that batter’s with good out-of-zone zone awareness are prolific at not swinging at unfavourable pitches.

An XGBoosted Decision Tree was chosen because the structure of the outcomes is clear and intuitive. A decision tree mimics human-decision making and with clearly defined outcomes and targets, the model can easily predict the run expectancy. Gradient boosting was implemented to improve the performance and efficiency of the model.

The features of the model are the following:

• plate_x: Horizontal position of the ball when it crosses home plate from the catcher’s perspective.
• plate_z: Vertical position of the ball when it crosses home plate from the catcher’s perspective.
• balls: Pre-pitch number of balls in count.
• strikes: Pre-pitch number of strikes in count.

As discussed earlier, these features are important drivers in determining the run expectancy of swing decisions.

The target of the model is the following:

• delta_run_exp_mean: As defined previously, the average run expectancy of each outcome given the specified count.

The models will be trained on 2020–22 MLB Data collected from Baseball Savant via the pybaseball python package. The trained models will then predict the expected run value of each swing decision (pitch) made during the 2023 MLB season to quantify a batter’s swing decision value.

Results

Each model had the following RMSE:

• No Swing: 0.045
• Swing: 0.294

The No Swing model seems to be performing much better than the Swing Model. Understanding the outcomes which were used to train the model provides further insight why this might be occurring. The No-Swing model uses two distinct outcomes with two distinct results. Under ideal circumstances, the result will be defined by the location of the pitch because the batter has now removed themselves from the equation.

For the Swing model, the batter becomes a part of the equation as they have decided to swing, however a lot of context of the swing has been lost in the process. Swing metrics such as attack angle and swing speed greatly impact the result of a swing. Additionally, the model uses location to determine the expected run expectancy, but these outcomes are not well-defined by only their location. Unlike pitches taken, swing outcomes overlap with one another in a great capacity. As seen in Figure 7, swings which result in hits into play and foul balls are clustered together. Additionally, all the swing outcomes can occur at any location, which can negatively impact the accuracy of using pitch locations to model swing decision run expectancy.

Figure 8 illustrates the distribution of actual and predicted mean run expectancies for each of the models during the 2023 MLB Season.

Figure 8: Actual vs Predicted Mean Delta Run Value Plot

The high accuracy of the No Swing model is captured well in the visualization. There are distinct spikes in frequency in both the actual and predicted distributions, which highlight the distinct outcomes of called strikes and balls with their respective count. The greatest discrepancies between actual and predicted values occur between the two large spikes, where the delta mean run expectancy hovers around 0. Since the model is a regression model rather than a classification model, there is a group of pitches which the model predicts that the not swinging will result in a neutral run value. Figure 9 illustrates that this group of pitches is located around the perimeter of the zone. This makes sense, as these pitches have a higher probability of not being accurately called as balls or strikes by umpires.

The Swing Model distribution in Figure 8 captures why using pitch locations may not accurately predict the run expectancy of a swing. Due to the overlap of outcome locations, it will be difficult for the model to accurately predict an outcome solely on pitch location and count. This is shown in the plot where the larger negative results (which represent field outs and strikeouts) create a substantial portion of the values. These values are not greatly represented in the predictions because of the lack of context which using only pitch location and count provides. While this may seem like an issue, these results make sense because of the multitude of outcomes that can occur when a batter swings. The important part is that the model defines areas which swings results in better outcomes, and those are shown in Figure 9. It should also be noted that swings generally result in a negative run value despite them being the only way to generate hits.

Figure 9: Predicted Delta Run Expectancy Hexbin Plot

In-Zone & Out-of-Zone Awareness

To quantify the value of swing decisions, each model was used to predict the run expectancy of each decision. The following metrics were calculated:

In-Zone Awareness — xRV/100 Pitches Taken

Batters which have good In-Zone Awareness are those who take fewer pitches inside the zone than others (e.g. batters who swing more often at pitches inside the zone)

Out-of-Zone Awareness — xRV/100 Pitches Swung At

Batters which have good Out-of-Zone Awareness are those who swing at fewer pitches outside the zone than others (e.g. batters who take more pitches outside the zone)

Decision Value — xRV/100 Pitches

This metric is the weighted average of In-Zone Awareness and Out-of-Zone Awareness.

Figure 10 illustrates how batters performed in each metric during the 2023 in each metric:

Figure 10: In-Zone Awareness vs Out-of-Zone Awareness Value Scatter Plot

Observations of the relationships:

• There are few batters which perform exceptionally well in both metrics
• A batter’s decision value can be carried by either their In-Zone Awareness (Corey Seager, Mike Trout) or their Out-of-Zone Awareness (Edouard Julien)
• Batters with high Swing% tend to grade out poorly in Decision Value

The last observation of Swing% in relation to Decision Value can be looked at further through another scatter plot. This is shown in Figure 11.

Figure 11: Swing% vs Decision Value Scatter Plot

The correlation between Swing% and Decision Value is -0.62, which means that there is moderately strong inverse relationship between Swing% and Decision Value. This result aligns with the real-life data that shows that swinging generally results in lower run expectancy outcomes.

This is a link to the spreadsheet which has the Decision Value Metrics for batters which faced 500 pitches during the 2023 MLB Season

An important Note:

• The metrics are graded on a 20–80 Scale, where 50 is considered average, and the Standard Deviation is 10.
• Decision Value is the average value of all pitches faced (Takes and Swings). A batter like Juan Soto may have worse awareness metrics than others, but his sheer volume of pitches taken greatly influence his Decision Value positively.

Conclusion

Analyzing swing decisions is important to quantify how effective batters are in their approach. Through this project, run values were assigned to different swing outcomes based on both the location of a pitch and the count which it was thrown. Batters who consistently make favourable decisions know which pitches to swing at or which pitches not to swing at, and the best batters know excel in both aspects. Quantifying swing decisions using only location and count data strips back a lot of context, but it helps to capture the batters which are efficient with their swings and takes.

Check out my GitHub Repository for the code and outputs of this project: https://github.com/tnestico/decision_value