College Baseball: Wins Above Replacement
Note from the editor, Sam Bornstein: The Iowa Baseball analytics team sponsored a project for four Master of Science in Business Analytics (MSBA) students from the Tippie College of Business. They were tasked with creating a Wins Above Replacement (WAR) metric for Division I college baseball. This article is written by the four students: Ben Ahlers, J.T. Cox, Aaron Dzaboff, and Jacob Swinton.
Introduction
Wins Above Replacement (WAR) measures a player’s value in all facets of the game by deciphering how many more wins he’s worth than a replacement-level player at his same position. A combination of many different metrics, WAR is used at the highest level of baseball to evaluate a player’s overall contribution and is divided into two subtypes: Pitcher WAR and Position Player WAR. For some context, in the 2019 Major League Baseball season both the American League & National League MVPs and Cy Young award winners were in the top 10 in WAR league wide.
Until recently, calculating WAR at the collegiate level had not been done. Iowa’s friends over at Driveline were the first — to our knowledge — to attempt this feat. The crux of the WAR statistic is defining a replacement level player and having widely available data to develop the metric. In MLB, it’s intuitive because there are readily available players in the minor leagues. With far greater variance in individual and team skill, and no readily available players to bring in, calculating WAR for college baseball becomes more difficult. In this project, we worked with the University of Iowa baseball analytics department to overcome these challenges, bring a WAR statistic to the collegiate level, and add another dimension to Iowa’s player evaluation system.
Data Gathering
The data used to create the NCAA WAR statistic was not easy to find — data collection has been a well-known setback for college baseball, including the complete lack of availability of necessary information. However, the majority of what you need is available programmatically through web scraping, via the baseballr package. By utilizing the R programming language, data on both position players and pitchers was readily available, as well as the scheduling and stadium information that fine-tunes the calculation.
Although, there were areas that data gathering was not so smooth. Fielding data and positional adjustment variables of Position Player WAR were unable to be gathered, and the same goes for the leverage aspect of Pitcher WAR. These are only minor elements of the final WAR calculation, but as more granular data is introduced at the collegiate level, the more specific and accurate this WAR metric will become.
Position Player WAR
There are several main elements that go into the calculation for a position player’s WAR value. These will be broken down in more detail below, but for now, here is the full statistical calculation that is used:
WAR = (Batting Runs Above Replacement + Positional Adjustment + Base Running Runs + Fielding Runs) / (Runs Per Win)
There are specific elements that make up each of the larger components in the above equation, many of which will be discussed below. In general, these are the main parts that figure into calculating a positional player’s WAR — notice how all aspects of their role on the field are included.
Batting Runs
The first and arguably most complex component of Position Player WAR is Batting Runs. Here is the hefty equation:
Batting Runs = wRAA + (lgR/PA — (PF*lgR/PA))*PA + (lgR/PA — (Conference non-pitcher wRC/PA))*PA
It seems overwhelming, but the methodology behind it is rather simple in the sense that it all starts with a player’s Weighted On-Base Average (wOBA). wOBA takes into account how often a player reaches base and how they were able to do so. This wOBA calculation is then adjusted for a player’s home stadium and the conference he plays in. Not only does this accomplish a statistical concept, but it makes logical sense as well — if a player competes against higher competition, he should be rewarded for doing so, compared to someone who plays against lower competition. Once the wOBA coefficient is properly adjusted, it is converted to wRAA — Weighted Runs Above Average. Finally, these values are scaled by multiple plate appearance metrics, serving as playing time adjustments.
Fielding
As mentioned earlier, fielding remained the anomaly of the data collection. Statistics such as errors, put outs, and assists were available during the process, but we did not feel that these measurements could adequately assess a player’s fielding value. Therefore, this aspect has been widely accepted as a pass-over until more data is available.
Baserunning
The baserunning aspect of a positional player’s WAR is commonly referred to as BsR, or baserunning score. Our version of BsR contains four sub-components: Stolen Base Percentage (SBP), Stolen Base Attempt Percentage (SBAP), Triples Percentage (TP), and Run Percentage (RP). These elements were discovered here are calculated as follows:
SBP = (Stolen Bases + 3) / (Caught Stealing + Stolen Bases + 7)
SBAP = (Stolen Bases + Caught Stealing) / (Singles + Walks + Hit By Pitch)
TP = Triples / (At Bats — Home Runs — Strikeouts)
RP = (Runs — Home Runs) / (Singles + Doubles + Triples + Walks + Hit By Pitch)
The BsR isn’t calculated from these sub-components, but rather it is derived using linear regression methods. As mentioned earlier in this post, the NCAA does not track the same advanced level of information that the Major Leagues do, which forced us to get creative. To handle this, we gathered MLB data that shared variables with our NCAA dataset, while having a BsR score to use as a target variable.
Using linear regression, we were able to determine weights for these sub-components as they pertained to an overall BsR score and proceeded to apply these weights to our NCAA dataset to create our version. Each score was calculated on a yearly basis along with the corresponding regression weights, which increased the accuracy.
Replacement Level
Replacement level is not as easily defined in the NCAA as its corresponding definition in MLB. For the Major League version of WAR, replacement level is a player that is readily available at the minor league level. NCAA programs do not have a secondary team to select players from, and even if they did, the talent would not be evenly distributed from conference to conference. Transfer-eligible players from D2, D3, etc. cannot be considered options since they are not readily available.
To standardize this across all NCAA conferences, a minimum of 10 games played was used to determine if a player was utilized as an everyday player or a bench player. As we mentioned, there are no readily available players in college baseball. Therefore, we might have to change our definition of “replacement” from acquisition-type evaluation to roster-management evaluation. We selected the replacement-level production to be the average runs per plate appearance of our sample of bench players.
Positional Adjustments
As mentioned earlier in the post, positional data was one of the areas that is not gathered on a specific enough level for WAR calculations. Position players are primarily listed in one of three categories: catcher, infield, or outfield. Whereas, the WAR calculation considers the specific positions (1B, 2B, SS, etc.) for its adjustments. The adjustments are used to properly scale to the offensive production that comes from each position. A soft-hitting first baseman cannot be treated the same as a soft-hitting shortstop, so the values need to be adjusted accordingly. Although we were unable to make these adjustments, we have provided the estimated values from Driveline. Once NCAA position player labels are tabulated more precisely, our WAR metric will be fine-tuned.
SS/C: +3 Runs
2B/3B/CF: +1.5 Runs
LF/RF: -1.5 Runs
1B/DH: -3 Runs
Pitcher WAR
Pitcher WAR operates differently from Position Player WAR in that there is only one dimension to calculating the metric, rather than three. So, Pitcher WAR is primarily an evaluation of a pitcher’s pitching statistics. It is derived using this formula:
WAR = [[([(League FIP —FIP) / Pitcher Specific Runs Per Win] + Replacement Level) * (IP/9)] * Leverage Multiplier for Relievers] + League Correction
Similar to FanGraphs, we chose to use Fielding Independent Pitching (FIP) instead of runs allowed per nine innings (RA9). Each pitcher’s FIP was calculated, then converted to the RA9 scale by adding the difference between conference average RA9 and Earned Runs Allowed (ERA), leaving us with “FIPR9”. Finally, this value was adjusted for the pitcher’s home park and the conference they play in.
Once FIPR9 was solved, it was used to calculate Dynamic Runs Per Win and subsequently converted to Wins Per Game Above Average. Replacement level was then factored in, with different coefficients for starters and relievers. As mentioned previously, it was not feasible* to consider the different leverage situations encountered by starting and relief pitchers. This mostly affects relief pitchers who pitch well in high-leverage situations, as well as those who do not. After that, all that was left was to scale the values to the number of innings pitched and add a final adjustment to ensure the WAR values are uniform.
While the leverage adjustment is the most severe limitation in calculating pitcher WAR, the lack of infield fly tracking in NCAA data served as another limitation. FIP is slightly more accurate by including batted balls of that nature. An infield fly is essentially a strikeout, but college baseball does not track the number of infield flies a pitcher produces. Therefore, college pitchers are not getting rewarded in that way.
*This is an area that can be further explored in subsequent semesters or years by the Iowa baseball analytics department, who have lots of exciting projects in the works.
Final Takeaways
College WAR is not a complete calculation yet, but the elements are available to make an accurate evaluation of a player’s value to their team. As the NCAA expands its ability to collect and provide public data, specific aspects in both Position Player and Pitcher WAR will be refined to increase the accuracy of the evaluation. Our initial research of FanGraphs’ and Driveline’s creations of MLB and NCAA WAR calculations, respectively, gave us a proper framework to follow to create our version.
Identifying the trends that exist from year to year between programs in the aggregate as well as players on an individual basis will be valuable to determine growth that is being made in the program. These values are also easily visualized in many formats, such as line graphs or box plots that provide a better viewpoint of the data as opposed to simply looking at the numbers.
Coaches, staff members, and student managers can utilize this metric to draw comparisons between players on their own teams, as well as on others. The creation of this metric for the Iowa baseball program has added yet another layer to the player evaluation system, and we are excited to see how the department utilizes and refines the metric!
