How I’d Fix the Sisk-Leiter Citation Studies

Like law review rankings, faculty citation counts are a little bit useful. Which means it is worth a least a blog post’s worth of attention to think about how to do them right. We should be grateful for the work that Greg Sisk and his team do in compiling the available studies, and to Brian Leiter for popularizing them.

The methodology could be better. It’s basically the same as the Washington & Lee Law Review rankings: an author gets credit for one cite for each article in which she is cited, regardless of how many times she is cited in that article. This tends to compress the distance between top scholars and others. For instance, I would say that I cite about 4.5 different Louis Kaplow articles in each article I write, while authors who cite me probably cite, on average, 1.1 of my articles. And, of course, it omits non-law sources, as well as some sources not covered in Westlaw’s databases (such as Tax Notes, which alone costs us tax folks about 10% of our cites, if I am at all representative).

What is more, raw citation counts omit some potentially useful information, such as the average number of citations per article, trends by year, and so on.

There is a remarkably simple solution to all these issues: just use Google scholar. Scholar’s coverage is not perfect (e.g., it too omits Tax Notes and other trade publications). But it does generate the H-index, a logarithmic-ish measure of impact (calculated as H number of articles with H citations — for instance, mine is 14 since 2011; Albert Einstein’s is 64 over the same period) and cites per year. The downside is that Scholar counts self-citations, which is largely why I have many more google scholar cites than Sisk reports.

So my proposal is simple. Put the burden on schools that wish to be accurately ranked to sign their faculty members up for a google scholar page. Then Prof. Sisk’s task would simply be to visit each page and add up the cites for each faculty. Objections, anyone?