Almost all of the arguments about the NIH cap are missing the point

Michael Hendricks
Jun 16, 2017 · 3 min read

I just can’t be bothered to post this as a tweet thread.

The NIH cap policy has taken a lot of criticism, some of it legit. But I think almost all of it misses the point. Let me explain. /1

The focus of the critiques has been on details of how inputs and outputs are measured and compared. Valid questions! /2

NIH has done analyses with multiple measures of input ($, GSI) and output (papers, cites, JIF, RCR). /3

All show diminishing marginal returns. Shouldn’t be surprising, because that’s just a thing that tends to happen. /4

It’s like measuring the height of NIH PIs and finding a normal distribution: yep, this thing is like most other things. /5

All of these I/O measures have their strengths and weaknesses. But more importantly, all show the same trend. /6

Where exactly diminishing returns kicks in and how fast it happens obviously depends on which measures you use. /7

So you can call bullshit on the log-log plot and find legit faults with the details of the GSI (which can be changed). /8

But that’s just cherry picking. It doesn’t change the consistent, overall underlying pattern that they found. /9

Use the Google and look at the last few years of data and communication on this instead of focusing on one chart/analysis. /9.5

BUT MORE IMPORTANTLY: “diminishing returns” is not the problem the NIH is trying to solve!!!! /10

The problem they are trying to solve is a toxically skewed distribution of resources. /11

They believe this skew is bad because it reduces the overall number of PIs (at a cost to ideas, serendipity, ecosystem richness), /12

and because the skew is highly correlated with career stage, which threatens the sustainability of the NIH workforce. /13

So, the argument is that de-skewing this distribution in some moderate, gradual way will help the NIH fulfill its mission. /14

The counter-argument is that there is a special benefit to resource concentration within a small coterie of senior PIs, /15

and that grant review is precise and accurate enough to optimally allocate resources to where they are best used. (inorite?) /16

NIH went looking for this benefit and could not find it. They found diminishing marginal returns, any way you slice it. /17

They also know enough about their own grant review system to know that it is not an optimal allocator of resources, particularly at lower paylines. /18

If there were data supporting resource concentration, they would show *increasing* marginal returns. No analysis shows anything close to that. /19

Arguing around the edges of the measurements or where the transition from linear to sublinear returns occurs is a red herring… /20

…because linear returns is almost as bad as diminishing returns as support for the argument that 10% of PIs having 40% of the funding is good. /21

So: the NIH needs to redistribute funding a bit, which means taking some from the tail and moving it to the middle. /22

My opinion is that there are no good faith arguments against doing this in principle, /23

which is why those making them often resort to denigrating struggling younger scientists with various versions of the riff-raff argument. /24

Perhaps, some say, a better approach would be to find the worst, dumbest NIH-funded scientsists and take their funding away? Sure. How? /26

It’s impossible, so that amounts to a do-nothing “more study needed” tactic. /27

So yes, some small % of people will take a moderate hit, NIH is trying to make sure it is the people who can most afford to. This is reasonable. /25

If you think GSI wouldn’t do that, then help make it better. They were more than willing to improve it. /28

NIH is good at consultation, but it won’t work if the community argues over details while a few influential insiders get their way. /29