The Psychology of NHL Draft Picks
Does the use of interviews and psychological tests in the Scouting Combine actually do anything?
In recent years the world of professional sports has begun to recognize the impact that an athlete’s mental state has on his or her performance, and with this the importance assigned to sports psychology has increased. This greater focus on the mental aspects of sport is evident in many professional sports leagues, and the NHL is no exception. In 2007, this trend led the NHL to begin incorporating psychological tests into their Scouting Combine.
The NHL Scouting Combine is a six day period used to assess the draft prospects of that year. As of 2007 it involves taking body measurements, doing physical assessments, and four to five days of interviews as well as a psychological evaluation. It is this last aspect of the Combine that reflects the growing awareness of the importance of the mental aspects of the game — but is the inclusion of interviews and psychological tests actually increasing the accuracy of team officials’ predictions of who will succeed in the NHL, or is it just a misguided attempt to not fall behind the times?
First, let me quickly run through how the interviews and evaluation work. The interview period allows team officials to meet with the prospects individually and ask them anything they feel like asking. Oftentimes psychologists are present for these. The goal of these interviews is to assess the personality traits of the prospects — to give team officials’ an idea of who will fit in best with the team, of what might hinder the performance of certain prospects, of who will be a leader, etc. Essentially, they are just job interviews with a psychologist present.
The psychological evaluation consists of two tests — one test that includes 220 yes/no questions and a second one that is simply a condensed version of the first and is meant to test the prospects’ reactions under stress and fatigue. This evaluation looks at 5 measures — mental toughness, coachability, validity, mental accuracy and mental speed. So, how has this all impacted actual draft picks?
Firstly, it is difficult to ignore the prominent finding that the predictive validity of employment interviews are modest at best (Marchese & Muchinsky 1993). Of course, there is the unlikely possibility that the validity substantially increases when you’re interviewing for the job of hockey player. Also, perhaps the validity increases with the input of a psychologist. The issue is, it really doesn’t seem like anyone has looked into whether the interviews conducted during the Combine have any sort of predictive validity whatsoever. This may not be surprising — employment interviews are the norm and employers tend to use them without asking questions — however, it is a problem when you take into account the fact that team officials seem to place much greater weight on these interviews than they do on the psychological evaluation.
But is it actually a problem? The thing is, there is also no evidence whatsoever that anyone has looked into the predictive validity of the NHL’s psychological evaluation either. Even in the creation of the evaluation, it seems that 5 measures were selected simply because it seemed to make sense that they would impact performance — but did anyone actually look into whether they do? If someone has, then no one is mentioning it. There have been several studies done showing that certain personality tests have strong predictive validity for the performance of NHL draft picks specifically, but there have also been many tests showing that other tests have essentially no predictive validity (Gee, Marshall, & King 2010). Moreover, none of these tests are the ones used by the NHL. Basically, there is no evidence that the interviews and the psychological evaluation do anything at all.
That being said, there is also no evidence that they don’t — there is simply no evidence. It seems that, unless someone has secretly looked into this and simply decided not to boast about how much the Combine has improved (an unlikely possibility, in my opinion), this is just another case of an organization deciding that something made sense and never checking to see what impact it is actually having.
______________________
Gee, C.J., Marshall, J.C., & King, J.F. (2010). Should coaches use personality assessment in the talent identification process? A 15 year predictive study on professional hockey players. International Journal of Coaching Science, 4(1), 2–3, 6–8.
Marchese, M.C., & Muchinsky, P.M. (1993). The Validity of the Employment Interview: A Meta-Analysis. The International Journal of Selection and Assessment, 1(1), 23–25.
Student number: 260456652
