Coffee and Closers: Predicting Relief Pitcher Performance
Part of my series on building a bot to manage my fantasy baseball team
Few people seem to have a good relief pitcher strategy, other than looking at saves. The prior post on predicting starting pitcher performance was rather straightforward, but predicting reliever performance based on pre-season projections is not nearly so simple.
We begin again by considering our correlation table:
What an unholy mess. Here’s the stories I glean from this.
A Close S[h]ave: Will Your Closers Save the Day?
Our league does not have holds, so relievers primary value is generally getting saves. One might initially be encouraged by the 68% correlation between save projections and the actual number of saves they get. But it falls apart if you look a little bit… closer [rim shot].
Sure, there is a correlation, but this is because half of the relievers considered are not getting any saves, and nearly three quarters will get five saves or fewer. So it’s quite easy to predict that any hurler picked at random will get very few saves, but it’s not going to help your season. If you are spending a high draft pick to get saves, you want to know if they will convert.
In other words, it’s not valuable to place a pitcher in the bottom left quadrant (low predicted saves, low actual saves). What you want is to predict the other three quadrants. Upper right is the closers that hit projections, upper left is sleepers, and bottom right is busts.
Therefore, we exclude relievers projected to get fewer than 5 saves and rerun the correlation table, we see the correlation between projections and actual correlation drops to just 29%.
Just a 29% correlation between predicted saves and actual saves. But we can improve on it fairly easily. Just as we found when examining starting pitchers, the best way to improve on this is the wisdom of the crowds. I could not find a better formula than simply adding a term for ADP:
You can check the results here, the graph on the left is the save projections, the graph on the right shows the adjustment from our formula above.
You can note it corrects for the biggest busts in the bottom right quadrant. These closers were projected by experts to get 30-some saves, but the crowds saw through this and drafted them late. The rule of thumb here is that it’s tough to get a good amount of saves for anybody drafted after the 200 ADP mark.
The save statistic relies very heavily on “inside baseball” — a talented closer could theoretically lose his job for going to third base with the general manager’s daughter, and the basic stats would have a tough time predicting this. I shaded the chart with “salary” (green being highest), as this was the stat I had available that was somewhat tethered to the inner politics. A team that pays $20MM for a closer will push to get their money’s worth. The t value was around 0.2, which may be predictive but not significantly so. Interestingly, the correlation was slightly negative.
The adjusted saves formula did a good job in the upper left quadrant (low saves projections, high saves) where it unearthed “hidden” gems like Corey Knebel (ADP 63), Felipe Rivero (88.1), and Raisel Iglesias (99.9). It did not catch Brandon Kintzler (483.3), Jim Johnson (556.7) or Brandon Maurer (536). Anybody who can find common threads between such players could use this to pick more hidden talent.
The adjusted formula did an OK job in the bottom right quadrant, where garbage closers who underperform projections dwell. It did well to weed out very low ADP relievers like Seung Hwan Oh (455.9), but was overly bullish on mid-range ADP guys like Mark Melancon (173.6), Jeurys Familia (177.7) and David Robertson (299).
Note that some closers in the 200 ADP range did quite well on the season relative to projections, such as Kelvin Herrera (200.1) and Fernando Rodney (234.1). How could one distinguish between these guys and garbage closers like Melancon, Familia, and Robertson? Salary may be an indicator — these guys performed very well at under $5.3MM, while the busts from above were making $7–12MM.
Relieved of Duty
Depending on the rules of your league, your strategy may be to not chase after saves and instead invest your energy into other categories. You may be obligated to start some relievers in your league, so this section focuses on how you might consider your strategy for other categories.
Innings Pitched
IP is an odd category for fantasy basebal. Even if this is not a category in your league, it is worth consideration because any reliever who sits the bench won’t be running up other incidental stats. Unfortunately, reliever projections on innings pitched are complete garbage:
If you want a better proxy for IP, look at an agglomeration of other projections surrounding IP. I present two possible formulas here.
I’m partial to the simpler one because I prefer fewer moving parts, but the more complex formula, which rests heavily on projected wins/losses as a proxy for likely innings, actually performed better on the holdout group.
Wins
Wins are exceedingly scarce, and throughout the season I put a disproportionate amount of work into chasing wins. Unfortunately, the typical relievers will get just a single win during a season, and the good ones might get about 3.
If you never get tired of winning where should you look? Oddly enough, the most predictive factor was projections of Innings Pitched per Game. I thought this was because of the effect of relief pitchers who got promoted to starting pitcher, but when I delved into the data this was not the case. It turns out that relief pitchers who were projected for greater than 1 inning pitched per game were the sorts like Michael Lorenzen and Tyler Lyons who were projected for ≥1.4 IP/G, few to no saves, and never ended up starting, yet still racked up 4 to 8 wins.
Another factor to consider is “reliability” which also looked to be a good predictor of wins and other reliever statistics.
Fortunately, most of these middle innings relievers have an ADP above 500, so they’ll be available on the waiver wire long after your draft.
Holds
I give this somewhat short shrift because our league does not consider holds. I did find it interesting that if you needed to predict this you could either count on the the projections (first formula), or get a marginal improvement by building your own formula (second formula).
The latter, an odd admixture of various counting stats, did slightly better on the holdout group.
ERA/WHIP
Similar to our analysis on starting pitchers, you can model pitching stats from projections, but using ADP is far better. For more data refer back to the prior post.
Strikes
I’m intrigued by the idea of packing your relief slots with strikeout artists. I don’t think it would be very feasible since typical starting pitcher averages 130 strikeouts on the season while relievers generally get about 70. Yet I can imagine a strategy where you grab starting pitchers with good ERA/WHIP but below average, and offset it with a bench of relievers in the top decile of K’s.
If you want to predict total strikeouts for the season, you should ignore the projections, as you can get better results with either of two simple models:
The first could be considered a model for “efficiency” — find relievers with a high strikeout rate who go for a lot of innings. The latter is to find young guys drafted highly who aren’t expected to get save situations. I preferred the first model for its simplicity and intuitiveness, but the latter performed better on the holdout group.
If you were drafting 3 relievers based solely on total strikeout projections, you’d get Andrew Miller, Aroldis Chapman, and Kenley Jansen and a total of 273 strikeouts. Our “efficiency” formula would whiff somewhat with Aroldis Chapman, Jacob Faria, and Andrew Miller totaling 248 strikeouts. Our young guys formula would have grabbed Corey Knebel, Felipe Rivero, and Brad Hand for a total of 318 strikeouts, though it would have chewed up a lot of early draft positions doing so.
