The NIH is responsible for almost all the basic research funding in the USA. Each year they spend $37 billion[1]. It dwarfs all the privately funded research institutes combined. Yet most scientists agree it is wildly suboptimal.
The problem is rooted in the incentive structures of the NIH grant selection and allocation process. Perhaps the best research on this is by Azoulay[2] who shows that the incentives of the NIH grant process make the funding at least 2X less effective at discovering breakthroughs than the incentives of the privately funded HHMI. Azoulay describes the dysfunctional incentives clearly:
The HHMI program has adopted practices that according to Manso (2011) should provide strong incentives for breakthrough scientific discoveries: the award cycles are long (five years, and typically renewed at least once); the review process provides detailed, high-quality feedback to the researcher; and the program selects “people, not projects,” which allows (and in fact encourages) the quick reallocation of resources to new approaches when the initial ones are not fruitful. This stands in sharp contrast with the incentives faced by life scientists funded by the NIH. The typical R01 grant cycle lasts only three years, and renewal is not very forgiving of failure. Feedback on performance is limited in its depth. Importantly, the NIH funds projects with clearly defined deliverables, not individual scientists, which could increase the costs of experimentation.
Azoulay continues to use rigorous methods to demonstrate that these differences in incentive structures make the NIH funded science 2X less effective at discovering major breakthroughs:
Our results provide support for the hypothesis that appropriately designed incentives stimulate exploration. In particular, we find that the effect of selection into the HHMI program increases as we examine higher quantiles of the distribution of citations. Relative to early career prize winners (ECPWs), our preferred econometric estimates imply that the program increases overall publication output by 39%; the magnitude jumps to 96% when focusing on the number of publications in the top percentile of the citation distribution.
Reading this paper’s methodology, you realize that the NIH wasting nearly half the tax payer’s money is the best case scenario. More than likely, it is much worse than that because this research only compares the absolute best NIH funded scientists (the people who won all the awards early in their career) compared to HHMI scientists. If you factor in the amount of money spent on mediocre scientists and incremental projects, significantly more than 50% of the NIH funds are wasted.
Wasting half of the $37 billion in tax payer money that gets allocated to science is a tragedy. The Unites States depends on scientific innovation to generate new commercial industries. If we could double or triple our rate of innovation, with the same amount of money, it would be one of the most important policy changes in US history.[3]
I think the right analogy for solving this problem is charter schools. Charter schools were developed as a way to test variations on the monolith public school system. A chunk of the taxpayer money that would otherwise go to public schools was given to a charter school instead to experiment on better ways to educate. By analogy: public schools are the NIH, private schools are private institutes like HHMI, and we need something analogous to charter schools.
You might have misgivings about charter schools. There is debate that while some charter schools are better than public schools, others are worse. With children’s education, this is ethical gray area, but with science this is exactly what you want! In science, you are looking for the outlier discoveries, and it is expected that there will be failures along the way.
With a “charter lab”, grant allocators could experiment with all kinds of incentives like:
- Investing in people vs projects
- Grant timelines
- Tolerance of failure
- Risky vs likely ideas
- Paying for staff scientists instead of grad students
- Engineering budgets to build infrastructure
- Large budgets for compute
- Universities lab vs separate lab
- Credentials or meritocracy
- Funding concentration (give more money to fewer people)
- Letting students make P2P funding decisions
- Given credit for impressive negative results
- Small vs large teams
- And so much more!
From these experiments with incentives, we could potentially radically improve the rate of innovation in science. At a minimum we know it can be doubled by simply using the HHMI incentives, but perhaps there is even more room for optimization. Maybe we can get 10X the number of major breakthroughs with the same amount of money. This is worth pursuing, even if the NIH is difficult to reform.
If you are fortunate enough to have an influential position that might be able to make this happen please email or DM me.
For general discussion and debate, please use the HN thread.
References:
