You’ve articulated very well lots of the reasons that I left academia. I looked at the gauntlet of fellowships and grants I’d need to get, did some simple maths, and decided I didn’t want to gamble my future on it.
This was about 10 years ago now. And the research funders knew then that the science funding and careers system was broken. The Wellcome Trust has been commissioning studies of where its funded researchers go for years to describe the attrition caused by this system. So we know that great scientists (and me) leave research because of the funding system. We know that long term, patient research (like, for example, the discovery of the structure of DNA) would probably not survive today, because of the funding system. We know that the attrition seems to be particularly acute for women, which contributes to the gender imbalance in science. And we know that the intense competition breeds bad behaviour, like faking results. We know all of this, and yet none of the research funders seems willing to do anything about it.
I agree with your recommendation for some kind of lottery system. We need to fund people, not specific projects, because funding projects makes it hard to ditch them when they’re not working out (which should happen in science). We need to rebalance funding towards long-term stable positions for mid-career researchers, and away from PhDs, who have become an exploited army of cheap labour, flogging themselves for long hours, little money, and relatively little chance of getting a permanent post.
I also think we need to make it easier for people to mix their research careers with other roles in practice, policy or industry. This would give people more personal career stability, as well as helping to get research into practice, and helping inform the research that gets done. But then I’m currently inching my way back towards academia as a public health person, so I would say that, wouldn’t I?
So the question is: if we know what the problem is, and people have got ideas to fix it, what will it take to get things changed? I don’t know the answer, but great articles like this must be part of the answer.