A (Re)Introduction to Consolidated Draft Pick Value

PDwhoa
PDwhoa
Jun 12, 2017 · 10 min read
Images from nhl.com

Previously, I introduced the idea of consolidated draft pick value, where I combined various ideas of the value of NHL draft picks into one value. Here, I revamp my previous work in an effort to arrive at a more useful value.

I explore the difference between results based analysis and GMs valuation of draft picks, as well as trading down at the draft, and differences in value by the draft round.

Using Eric Tulsky’s model as an approximation of GMs valuation, there is a large gap between what they are willing to pay for draft picks versus what draft results suggest they should be worth. This discrepancy could be an indicator of an undervalued and advantageous market for smart managers.


Background

The first version of this was made up of ten interpretations of draft pick value. In my eyes, this version had problems, so I am hoping to rectify them or minimize the errors with this version.

For instance, Steve Burtch modeled with three different games-played cutoffs for a successful pick, and all three were used and I felt this might have given his measure too much weight in the grand scheme of things.

Chemmy’s model was Jibblescribbits’ model extrapolated to a larger scale. This is just counting the same model twice and has been corrected here.

In this version, I stuck with models that were based in math. Set parameters, set limits etc. etc. For example, Scott Cullen assigns more value to “generational” players drafted over “elite” players, neither of which have a defined criteria. The subjective approaches might be good and might add insight, but they also might not and I have no way of knowing which does which.

I also missed a model from Dawson Springus, probably better known as DTMAboutHeart on Twitter (or DTM in here for short).


The Models

After taking out some repetitive methods and adding the one I missed, I was left with 5 different approaches to draft pick value.

I encourage you to read them all as they have different approaches but I’ll summarize.

Michael Schuckers wrote a paper on the topic, using games played as his stat to measure a successful draft pick. His findings appear fairly consistent with similar methods.

Eric Tulsky used a completely different method. Instead of looking at the success or failure of drafted players, he looked at how GM’s valued the picks through trades. This led to valuations that were wildly different than the others, with much more value at the top of the draft and a far larger gap between the first and last picks. This indicates that there’s a large divide between what GMs think picks are worth and what history tells us picks are worth (more on this in the analysis section).

Jibblescribbits used time on ice instead of games played and came to conclusions that line up with the others, save Tulsky.

DTM used Point Shares, an earlier attempt at a catch-all stat, to determine worth from the draft. An interesting wrinkle here is that he restricted the measure to a player’s first 7 years, as that’s how long a team has control contractually from the entry level deal and restricted free agency. Even with this added restriction, the results came out similar to the others that used draft results to find values.

Burtch looked at the probability a player would reach a games played threshold based on their draft slot. He did this with three different thresholds (60, 100 and 200 GP) that had similar (but not identical) results. For this work, I used the 200 GP threshold, which was an arbitrary decision on my part.


Methodology Notes and Other Thoughts

A few more notes before I dive in. First, not all of these articles had values for all 210 picks. Only Schuckers and DTM had individual values for each pick. Burtch had an equation of a line in his graph on Sportsnet, and I used that to find his values. The other two had values given to ranges of picks. For these I just averaged the descent from one range to the other. So when Tulsky has a value for picks 10 and 12, but not 11, I took the gap between the two and evenly distributed it between the missing picks. (Shout-out to Chris [@crzycanucklehed] for bringing that simple math to my attention.) Tulsky’s data only had values for 35 picks. So to fill the gap between 10 (36.5) and 12 (32.9), the difference of 3.6 points was split in half to give pick 11 a value of 34.7. When looking at the missing information between picks 105 (0.64) and 120 (0.4), the gap of 0.24 was divided by the 15 missing selections, leading to a gradual decrease of 0.016 every pick.

With the addition of Las Vegas to the NHL, there are now 217 picks in the draft, and most of these analyses only accounted for 210. I extrapolated their values similarly to how I did missing values above. Again, this was arbitrary and isn't 100% mathematically accurate but it’s the last 7 picks of the draft, and the values are lower than the higher picks, so they pass my eye test.

Speaking of Chris, he has a neat spreadsheet that allows you to look at this year’s draft through the lens of many of these different models, while also providing information about basically anything you might want to know about the picks, including who owned them originally and how they ended up where they are now.

Also, Namita (@nnstats) gave a wonderful presentation about perfect drafting at the recent VanHAC. Read her slides because it is a concept that is fascinating to me (and was inspired by the infamous Boston draft in 2015).

Since all five of the models used different scales to determine draft pick value, I had to normalize the scales before I could consolidate them. To do this I found the amount of value each pick had compared to the total value in the draft. I used a percentage to determine this, so if Made-up Model said the 1st pick was worth 60 points and the combined value of every pick was 100 points, the 1st pick according to Made-up Model would be worth 60% of the draft.

From here I could combine the percentages from each model and voilà, consolidated draft pick value. This is where the previous consolidation stopped. Each pick had a percentage attached to it and that was that. But personally, I found that tough to wrap my head around. Saying the 1st overall pick was worth 3.46% sounds complicated, but saying it has a value of 346 is less so. The only additional step is multiplying all the percentages by 10 000, hopefully making draft pick value easier to grasp.


Consolidated Draft Pick Value and Analysis

Now, the values themselves.

The spreadsheet linked here has 220 individual values that make up Consolidated Draft Pick Value. I won’t focus on pick vs. pick too much, but feel free to do that yourself if you have the time. I will take the time to look at the top picks though. These values appear to line up with conventional wisdom: 1st separates itself from the rest and 2nd distinguishes itself as well before the values start to cluster together. The gap between 1st and 2nd is the same from 2nd to 5th and 5th to 12th. From another angle, the gap between the top two picks is about 64, but by 10th and 11th that has shrunk to 8, and in the 20’s it’s between 3 and 5. The very top of the draft has the value and it drops off quickly.

Dividing picks into rounds has always struck me as a little weird because deciding 31st and 32nd overall are somehow vastly different, or more different than 32nd and 57th doesn't make sense. However, teams own and trade in the (at the time) unknown values of picks divided into seven rounds, so I’ll look at some differences there as well.

Value is the sum of the 31 picks in that round

Nearly half of all the value to be found in the draft is in the first round. The second and third rounds hold the majority of value that is left with value gradually declining in round four through seven. Interesting to me is the lack of difference in the last few rounds. It seems to me that a knowledgeable person could use the gap in name value and the lack in real value to their advantage. A fifth sounds more impressive than a seventh, but swapping those could leave a gap that is not difficult to make up.

On a similar train of thought, I think teams undervalue those late rounds (more on that later) and a team could pull an EA Sports special and ask other teams to “sweeten the value just a touch” by throwing in late round picks. A second rounder for a rental; why not a second and a sixth or seventh? A caveat here is we as spectators have no idea how the negotiating process works in these situations, so maybe this is not a feasible as I’d like to think.

Back to the GM’s and their value for a minute though. I mentioned that Tulsky looked at market value instead of draft results like all the other models included here and I want to take a closer look there. To recap: Tulsky looked at pick-for-pick trades and used them to inform his version of draft pick value.

I think this can be used as a suitable proxy for GM’s valuation of picks. I don’t think these analyses are at the level where we can claim that GMs are wrong. All the current methods of determining what a successful draft pick is are flawed, and instinct tells me that early picks should have substantially more value than the others, but it’s just as likely my instincts are bad too. At the very least it is interesting to note the divide between what GMs are paying for picks and what the results are saying those picks are worth. That said, what happens if we compare the results-based values to the market-based values?

The first-round jumps off the page. Over three-quarters of all the value in 31 picks. The second round is where there is the closest agreement, followed by the seventh but even there the margins aren't small. Every round outside the first appears underrated by GM’s, while the first seems massively overrated here. Again, this could be an opportunity to add some value at the margins and increase a team’s chances of hitting in the late rounds. [1]

Last, as is always the question when draft pick value is mentioned, is about trading down. In the NFL Bill Belichick made this approach famous (or so I’m told), consistently trading down and continuing to win. No one in the NHL has embraced it like him, but at every draft teams swap picks. Looking back at the 2016 Draft, teams that traded down consistently came out with more value. It is important to realize why teams trade up. It is related more to specific players who teams think will exceed the expected value in that slot.[2] Still, there is no guarantee that the targeted player turns out or exceeds the expected value (just ask Tyler Biggs), making trading down an attractive and possibly underutilized tactic on the draft floor.

[1] As a side note, Tulsky’s work was done in 2013, using trades from 2006–12 and I think it could be updated by someone with some math experience.

[2] For a thorough look at why teams trade up, Arvind at PPP wrote about it here.


TL;DR and Conclusion

For the too long, didn't read crowd, I felt my original consolidation had some flaws and I was unsure if some of the methods included were worthwhile, so I attempted to fix the flaws and trim down the valuations included. I also added a step to the consolidation to end up with numbers that are hopefully easier to comprehend than the percentages from before.

For those wondering what their team’s outlook is at this draft, do not fear. Once the Cup has been awarded and the draft order finalized I will post the 2017 Draft Pick Value on my twitter page along with some scattered thoughts about it.

In terms of differences between old and new, there’s a larger distribution of value from top to bottom, with 1st overall now worth about 70 points more than before.

The top of the draft has the most value and drops off quickly, with value starting to blend together even in the top 10. The first round has nearly half of all the value in the draft, with the second and third rounds making up most of the remaining value.

Using Eric Tulsky’s model as a proxy for GM valuation, there were some interesting trends, including massive perceived value in the first round, and very little in other rounds, especially the later ones. This could mean that the relative undervaluing of later rounds could be exploited by a knowledgeable GM.

This updated consolidation still supports trading down at the draft, with teams trading down in 2016 consistently recouping more value.

This revamped Consolidated Draft Pick Value better reflects various mathematical interpretations of draft pick value, and still encompasses varying approaches, resulting in a better consolidation than before.


A quick thank you to Achariya and Alan for the extra eyes before this was finished.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade