The Best Baseball Statistic

A Simple Question. If you had to choose one statistic that best describes a player’s overall contribution to winning, what would it be?

Wikipedia defines more than 117 different baseball statistics that are possible candidates. Because players’ participation can be separated into offensive and defensive components, almost all statistics focus on either offensive or defensive contributions. The challenge is to define a statistic that combines these two types of performances in a fair and justifiable way. That is the subject of this post — it defines five necessary attributes that such a statistic must have.

5 Necessary Attributes Of The Perfect Baseball Statistic

A perfect baseball statistic would provide a single number that summarized the total contribution of a player. No claim is made that these five characteristics are sufficient — only that they are necessary. Furthermore, because every statistic is calculated using historical data, predicting future performance with absolute accuracy is impossible.

Does a perfect statistic exist? Not by a long shot. Most statistics are player-based. Some are team-based. Most measure one of the four components of on-field performance, namely: batting, running, pitching and fielding. Several aggregate measures of performance have been defined that combine more than one component — Linear Weights, Wins Against Replacement and its successor OpenWAR are three. An excellent discussion about weaknesses of the classical stats can be found in many references. One of the best is Smart Baseball by Keith Law published by Harper Collins in April 2017. However, no existing statistic has all five attributes described in this post. The acronym CRAZI can be used to remember the five characteristics. An ideal statistic must be Comprehensive, Run-based, Additive, Zero-Sum, and Independent. Explanations follow.

Comprehensive. The statistic must be calculated from all the events in which a player participated — not just the offensive ones or the defensive ones. The statistic must not only measure how the player participated in an event such as a stolen base but also measure the quality of his/her performance in whatever role the player had in the event. Quality must be measured in terms of how the player performed relative to all other players who had the same role in the same situation. Statistics like Batting Average, Saves, or Caught Stealing are not comprehensive.

Run-based. The team objective in every baseball game is to score more runs than the opposing team. Therefore, the perfect statistic must be measured in runs. That is, in each play did the player increase or decrease the number of actual or potential runs resulting from the play. The question “by how much” must be answered by using the importance of the role of each participant in a play. The importance of roles is discussed in the next section.

Additive. In every game, the team statistic must be the sum of the players’ values. If valid, there will be a perfect correlation between the team statistic and the game result. That is, the team that wins the game will have a higher statistical value.

Zero-Sum. Baseball is a zero sum game. What is good for one team is equally bad for the other team. This means that in every play the sum of the statistic values of the participating offensive players must be equal and opposite to the sum of the statistic values of the participating defensive players. The simplest example is a strikeout. The pitcher should get a plus value representing the number of potential runs prevented. The batter should get a negative value having the same magnitude representing the loss in potential runs.

Independence. The fifth necessary characteristic is that each player’s statistic value in a play must be independent of factors beyond the player’s control. A statistic such as Runs Batted In is clearly not independent of factors such as runners on base, the venue of the game, the player’s position in the batting order, etc. Independence can only be achieved by comparing a player’s contribution with every other player’s contribution in the same set of circumstances. Fortunately, the extensive volume of detailed, historical play-by-play data means it is feasible to determine parameter values needed to calculate the statistic. If a player’s performance is better than the average performance, the player should get a plus value for his participation.

Challenges

1. Subjective Valuations. On-field performance is the most important evidence of a player’s ability. However, several other hard-to-quantify factors are also significant. Examples are injuries, age, attitude and trends. A perfect statistic would include these factors but would be very difficult to measure. Getting agreement on the relative value of subjective factors is an even greater challenge.
2. Data Variations. The most important parameter when calculating potential runs in a half-inning starting from a given state (the state is defined by the number of outs and runners on base) is the average number of runs that occur starting in that state. For example, in the 2017 Major League season the expected number of runs when there is 1 out and the bases empty was .275 runs. These expected runs values have deviated very little over the years. However, in other leagues they may be different and need to be calculated from historical data. Any methodology that claims to be “perfect” needs to initialize the league-dependent parameters. If parameter values have significant variances, then the resulting statistic can only be expressed as a range or probability distribution. Similarly, factors such as the park effect, day/night, turf/grass, open/closed roof, defensive positioning and others may be deemed important. If so, the calculations need to take these in account.
3. Role Importance. How important are the roles of the pitcher, the catcher and the fielder in a stolen base event? Although there may be a consensus about the relative contribution of these defensive positions in an event, specific degrees of responsibility must be assigned to each role in order to calculate participating players contribution to the result. Any worthwhile implementation of the perfect statistic must use default values that are reasonable for each type of baseball event. These responsibility percentages must allow edits to reflect outstanding player performances in specific events.
4. Relative Importance of Offense and Defense Components. At a higher level, the relative importance of batting versus running and of pitching versus fielding needs to be defined. Because of the zero-sum nature of the game, offense and defense are equally important. However, how important is running compared to batting or pitching to fielding? Given sufficient data, these parameter values can be determined. These values are required to calculate the total performance of players who may have specialty roles on a team — pinch runners or designated hitters for example.
5. Bias Against Defense. A perfect baseball statistic requires that the on-field performance of all players regardless of the position played, can be measured with a single statistic. Because the statistic is run-based and because runs are scored in every completed baseball game (unless a league permits zero-zero ties), players whose role is primarily defensive — pitchers in particular — will have lower statistic values than players who contributions are mainly offensive. Without correcting the inherent bias against defensive roles the statistic cannot provide a fair comparison of offensive and defensive performances. This bias is independent of the requirement that the stat values in every play sum to zero. It is simply a reflection of the fact that players typically do not have equal offensive and defensive involvement in plays. Eliminating this bias in the algorithm used to calculate the total performance of each player is not a trivial task. When accomplished, a plot of the player stat values will have the normal bell shape and allow a fair comparison between pitchers and hitters for example.
6. Data Sources and Granularity. Many web sites and organizations provide detailed information and tools for capturing data needed to calculate a statistic that embodies all five attributes. Major League Baseball provides detailed textual descriptions of every play in every game. The Statcast product for example captures a terabyte of data for each Major League game. The data includes extremely detailed information such as the spin rate of each pitch, the exit velocity of the ball off the bat, and the delay of a fielder’s movement toward the ball. Although a perfect statistic might be enhanced by this level of detail, the fundamental requirements are knowing the state-to-state information for each event in a game. The definition of “state” can be augmented with many dimensions but the cost of data capture and the sparseness of state-to-state transition probabilities makes this enhancement difficult to justify.

Uses of the Perfect Statistic

The perfect statistic defines a single aggregate measure of on-field player performance that allows all players — regardless of their role — to be compared fairly using a single numerical value. Three of the important uses of the statistic are:

1. Fair Salaries. The salary a player deserves based on his on-field performance can be calculated using the statistic’s value and the total payroll of the team. By comparing all players on a single statistic it becomes much easier to determine salaries that are justified by on-field performance.
2. Team rosters can be compared because the team statistic is the sum of the players’ statistics. This means that good decisions can be made regarding player trades and roster changes. Roster totals can be used to forecast league standings.
3. In-game decisions regarding lineups and player substitutions can be improved. The classical statistics are still useful for assessing specific aspects of performance. None, however, can claim to be the best composite measure of a player’s value.

Does the perfect statistic exist?

RunPlusMinus™️ (RPM) is the only known statistic that satisfies the 5 necessary CRAZI requirements described previously . It also solves the 6 challenges described above.

What is the RPM statistic? In simple terms, each play in a game results in runs scored and/or a change in the potential for the offensive team to score runs. The sum of these two values is compared to the historical average of all plays that started with the same number of outs and base runners. The difference is a plus or minus value and is measured in runs. This plus or minus value is divided among each player that participated in the play. A player’s share depends on his responsibility for the outcome of the play. These player plus/minus values are the values of the RPM statistic for each player involved in the play. The total of the player RPM values in every play is zero. They can be added to give a player’s above or below average performance in the game. The team’s RPM value is simply the sum of the players’ RPM values.

We have made calculations and generated statistics for the 2017 MLB season. Over 48 million data points have been used to calculate player and team performances.

Shown below are three outputs from analyzing the 2017 MLB regular season games. (Including playoff games would bias the results in favor of players who participated in the post season.)

First, the graph below shows the number of players that have a given performance rating. It demonstrates that the average player performance is zero. Additionally, the shape of the curve indicates that the number of players with performance ratings above and below average is essentially the same. Furthermore there are relatively few high and low performers.

A bell curve of player ratings for the 2017 regular season

Second, we claim that the winning team in every game has a positive RPM total. This claim is validated in the chart below. For each possible winning run margin (1 to 18) it shows the high, low and average RPM total of the winning team. The line shows the strong linear relationship between run difference and the average winning RPM value.

Third, who were the top 25 players in 2017? The chart below shows those with the highest rating (the weighted sum of batting, running, pitching and fielding ratings). It also shows the players’ salaries and the salaries justified by their on-field performances (what % of the total salary pool did their performance earn).

Top 25 players in the 2017 regular season by RPM Rating

It is difficult to find a table of the top players that isn’t based purely on expert opinion or limited to one of the four components: batting, pitching, fielding and running. RunPlusMinus™️ allows for the easy statistical comparison of total performance (column 3 in the table above) which is derived from the weighted sum of the four component performances. A variety of other RunPlusMinus™️ reports show rankings by team, by league, for subsets of games, players etc. that provide useful insights that can help settle debates about who is more valuable than who.

To request more information on RunPlusMinus™️ and to learn more, visit us on our website: RunPlusMinus.com