Survivorship Bias in Climbing Talent Development
Survivorship Bias is the mental error of only concentrating on the projects or people that have been successful, and overlooking those that failed when analyzing what made something a success.
The error is to focus on things that survived when you should really be looking at things that didn’t. History is written by the victors. But concentrating on “survivors” and systematically ignoring “casualties” of any situation leads to distortions in our conclusions.
We overestimate the odds of success because we only hear about the successes!
A famous example of Survivorship Bias in action comes from military history. During World War II, statistician Abraham Wald was working for the U.S. military to try and figure out where planes should have their armor reinforced, in order to avoid getting shot down. The military’s initial efforts weren’t as successful as they’d like, and Wald knew why.
The military had decided to only reinforce those areas where planes had been shot. But the problem was, they were only seeing planes that had returned. In other words, where these planes had been hit was survivorable damage, because they’d flown home. The planes who’d crashed hadn’t returned and therefore hadn’t brought back data about the places where the damage proved fatal.
Wald proposed that the military reinforce the areas where the surviving planes had not been shot, as those were the places where downed planes had been damaged. Wald’s brilliant observation was correct, and saved many planes from a crash landing.
Survivorship bias — does it happen in climbing development?
Are we Fooled by the Winners? Do we spend too much time focusing on the few that make it rather than the many that don’t?
History is written by the victors. But concentrating on “survivors” and systematically ignoring “casualties” of any situation leads to distortions in our conclusions. We overestimate the odds of success because we only read about the successes!
Often, the sampling and explanatory cherry-picking means that “almost any feature of interest can appear to be associated with success” as long as there exist at least some examples of such an association.
In 2015, the International Olympic Committee released a consensus statement raising concerns regarding recent trends in youth athlete development.
The statement questioned the validity of talent development models in youth sport, while also referring to the problematic nature of early specialisation, parental pressure, coaching styles, media sensationalism, and the view of youth athletes as commodities. Further, the IOC consensus statement recommended that frameworks for youth development need to be more flexible, incorporating a combination of both best practice and experience underpinned by quality up to date research.
Read the IOCs a consensus statement here:
The search and pursuit of the alchemic process of how to unearth and develop professional athletes has been described as the ’holy grail’ of any sporting nation.
In climbing, we don’t have environments like in football, where developing professional players enjoys both increased profits and recognition.
In other sports, structured performance pathways across countries are relatively homogenous and commonplace, with many clubs and organisations investing heavily into the professionalisation of the identification and development of talent.
Research has highlighted inherent problems related to the predictive value of future performance through early identification. For instance, Coaches’ talent identification is underpinned by an ‘intuitive’ approach (what feels right!).
This is of course true for me too.
BTW, I’m currently producing a climbing performance coaching course by the same name, should you be interested.
So, while I pride myself of my coaches eye, I still have to agnowledge that what feels right is greatly influenced by my experience of previous identifications, personal interpretations of what the particular climbing activity entails, and the coaching culture in which i find myself.
As any experienced coach I’m therefore aware that early indicators of performance are often poor predictors of future performance.
One example of why subjective methods for talent identification are criticised is due to a bias towards the selection of players born earlier in the (age category year), a phenomenon that has been referred to as the Relative Age Effect (RAE).
In many sports an earlier maturation can affect the development of several anthropometric and physiological variables biasing the selection of ‘older’ players, there are inherent problems with frameworks where each stage is associated with a chronological age.
It has been argued that children disadvantaged by birth date or physical maturity might have become equally skilled senior athletes if they were afforded equivalent developmental opportunities.
In climbing we see this in the youth categories with taller athletes that can reach holds that the smaller ones can’t. On a side-note, in my book that’s poor route-setting, as I probably mentioned before.
Nowadays in climbing though we often see a reversal of the RAE, since competitors more and more seem to be over their peak even already in their early twenties.
Federations and clubs loose climbing talent because of burnout in an alarming rate. I havn’t looked into the statistics, but the average age in finals is on a decline.
A related issue I’m constantly asked about by other coaches is if they should ability group their practice.
Often it is even demanded by parents as they believe only likewise talented kids will help the development of their kid.
Indeed is the practice of early selection and de-selection of children through ages and stages are now central tenets of player development programs around the world.
Often pyramid like in structure, these type of development programs are seen as the Standard Model of Talent Development. Lacking in both empirical and conceptual validity this model is based on the presumption that development and performance in sport are conceptually linear and predictable.
I already mentioned the International Olympic Committee consensus statement on youth athlete development from back in 2015, highlighting possible negative influences on health and well-being.
Youth sports has become both adult and media centered nowadays
It was suggested that the ‘culture’ of youth sports in general, has become disproportionately both adult and media centered, viewing youth athletes as commodities promoting a sensationalism that has an influential grip on adult expectations.
Indeed, language plays an important role as does acknowledging that learning and development cannot be fully understood without taking in to consideration the environmental, historical, and socio- cultural constraints that can influence learning and development. For example, many resilient beliefs and even the attributes and skills appreciated in young climbers are culturally embedded in traditional pedagogical approaches, organisational settings and structural mechanisms founded upon specific socio-cultural and historical constraints.
Talking of the importance of language, one word in particular makes me cringe when it is used in youth sports.
The word is Work!
She earns it, she has put so much work.
All the hard work is paying of.
Listen to athlete co-commentators on a competition livestream nowadays and you hear a lot of these phrases. Semi successful climbers call themselves professional athletes, even when they’re happy to receive free products and are relying on their parents or other support systems outside of climbing.
This creates all kinds of dilemmas that are outside the scope of this show, for example: I worked so hard, I surely deserve to succeed, right?
Well…
Many coaches still firmly believe in linearity of human development, — that you need to put in the hard work, i.e. drill in the foundations and prepare your body, before you are even allowed to play and enjoy yourself in climbing.
Research shows quite the opposite being true — If we really want to support the well-being and foster more and better climbers, then it is important to consider the complexity and non-linearity of human development. This requires an understanding of what learning and development is and what factors that can influence it.
above: the female juniors lead world champions from 2017, none of them is been seen five years later at the open finals.
Emphasizing on Puritan work ethics and an obligation to consistently work diligently is not helping here I think.
This is why we have to reflect on our believes and check in on our survivorship bias.
One problem i see is that often so called talent development models are legitimised and justified through a ‘copy and paste’ template of talent development models imported from ‘successful’ nations or clubs.
There is clear reason to doubt whether these models of talent development and organisation in many elite sporting structures have sufficient scientific validity. There is even further reason to reinforce issues associated with the uncritical “copy and paste” of best practice from successful countries or clubs.
To better understand player/athlete development in and through sport, culture and context do matter.
For me for example the timespan athletes are competing is only one aspect of what i hope climbing contributes to their life. My idea of performance sports is:
* as many as possible,
* as long as good and
* as good as possible.
Here is an interview the IFSC did with me back in 2017.
Perhaps it is time to investigate ways to comprehend the distinct contextual complexities of cultures, communities and situations to support a broader perspective on athlete development.
For now I’m gonna leave you with these questions:
“When I look back to see what’s gone right, have I also looked at what’s gone wrong?”
“Am I accounting for the features, tendencies or characteristics of failure as well as success?”
“What features or choices do success and failure have in common?”
If you found these thoughts interesting, please leave a comment below and subscribe to the channel.
Thanks for watching and see you next time!If you find these considerations interesting, please check out the educational resources here: https://udini.com/education/
