The h-Index Is a Lie: A Bibliography of Truth
Those of us who are junior researchers in academia, particularly science, are encouraged to focus on the qualities that will get us hired as faculty at the institutions of our choice. If we want to end up at a teaching-oriented institution, we should get in as much teaching experience as possible, and if a teaching certificate is available at our grad or postdoc institutions, that won’t hurt. If we want to end up at a research institution, well then, fuck teaching (no really, don’t do it, it will hurt you) because no one gives a fuck about whether you know how to teach (no seriously, don’t teach, it will hurt you; no, don’t worry about how your lack of practice will harm your future students). Instead: worry only about publishing. Publish whenever possible. Publish garbage, if necessary. Publish the most boring, low-hanging fruit you can find. Ideally you will publish the solution to quantum gravity, or at the very least something trendy enough to appear to the status quo to be paradigm shifting. But publish something, even if it’s just about participating in what is essentially a glorified conceptual popularity contest. Getting in line with what is popular is much more important than following your scientific nose.
Why? Because when people aren’t counting the number of publications on your CV in tandem with the prestige of your adivsers and the instutions you are affiliated with, they are looking at something called the Hirsch index, or the h-index. The h-index is a mathematical function that is essentially determined by how many papers a researcher has published and how many citations those papers have received.
An index like this is considered to be one of the most important features of a researcher’s reputation. In fact, entire careers can be boiled down to this one number. And it is — against logic — considered to be a neutral measure of a researcher’s productivity and therefore their capacity to succeed as a professor.
Never mind that a professor at a research institution will spend a significant amount of time teaching and supervising students. That if they are a person of color, especially a marginalized gender minority of color, they will spend extra time trying to help marginalized students who are being eviscerated by white supremacist academic environments. That the h-index doesn’t measure preparation to do these things and continues to be important at best signals that academics live in a fantasy land where publication rate is truly a proxy for high quality of productivity and citation rate is truly a proxy for impact of one’s work, rather than impact of one’s pedigree.
At worst, and I admit to believing the worst here, it signals that our white supremacist university campuses just don’t care about marginalized students and the different qualities a heterogeneous research force could bring to the paramount scientific questions of our time. And I believe our funding agencies enable them by not reconfiguring granting structures so that it is harder for universities to not care.
The h-index encodes a. that the hard to quantify qualities that underrepresented group scholars bring to the table don’t matter and b. an inability to recognize that not all publication records are made under equal circumstances and that given healthier and well-resourced environments, numbers might improve. Perhaps institutions should consider truly giving people from marginalized backgrounds an equal chance — a safe environment, the resources they need, the mentors they need, the money they need, and some relief from the pressures that come with being marginalized — and then seeing what we can do with it. The fact that we do all that we do, survive all that we survive, and still publish at all is in fact a major qualification. Imagine what we would achieve if we were given the chance.
I’ve noticed that some public institutions rely rather heavily on something like the h-index, which to me flies in the face of their job as public institutions, which is to be broadly accessible. I think additionally, not interrogating the use of citation number as the paramount feature of hiring allows scientists to continue to think that their hiring and admissions processes are both rational, neutral and not predicated on overtly and covertly racist precepts and social structures, even though in reality they are deeply entwined with them.
Besides my observations, there’s also literature to back this up. Publications! That are probably being counted in some people’s h-indices. The lesson that I take from these publications and my own experiences is the one that Black students across the U.S. have been trying to teach their white professors this fall: business as usual is a failure and that means the academy needs to upend its traditions in exchange for ones that reflect a commitment to equity of opportunity in the classroom. It is time for academics to do their jobs and ask questions, to interrogate traditions and create new ones.
The number one demand across the country, by the way, is to hire more minority faculty.
(It also seems to be that Google is really pushing the h-index as a means to centralize their app Google Scholar, but corporate disruption of academic matters will have to be another blog entry.)
Here is a live bibliography (much of which is behind a paywall, sorry):
“As noted in your News story, tallying how many papers a researcher publishes (their productivity) gives undue merit to those who publish many inconsequential papers.” — letter to Nature
“Prolific writers are disproportionately likely to be white males because the primary criteria used to define productivity, quantity of journal articles and citations to them, reflect career paths, work assignments, interests, and access to resources that are much more characteristic of white men than most women and minorities. This suggests that, in addition to examining the question of whether traditional productivity criteria are equitably applied, it is essential to examine the question of whether productivity criteria are equitable.” — U.S. Department of Education Report
“Cumulative experiences with discrimination and stereotypes may partly explain higher attrition and lower publication productivity among blacks and Hispanics.” — Disparities in publication patterns by gender, race and ethnicity based on a survey of a random sample of authors
“We conclude that the h index, like many other metrics, may reflect systematic gender differences in academia, and we suggest using caution when relying on this metric to promote and reward academic psychologists.” — Gender and the h index in psychology
“Data from radiation oncologists show a systematic gender association, with fewer women achieving senior faculty rank. However, women achieving seniority have productivity metrics comparable to those of male counterparts.” — Gender Differences in Publication Productivity, Academic Position, Career Duration, and Funding Among U.S. Academic Radiation Oncology Faculty
(“In other words, if you just assimilate, it will all be fine. If assimilation is not your thing, you probably don’t belong, sorry.” — White Supremacist Patriarchy)
“Moreover, studies of gender schemas suggest that evaluation biases will infiltrate every evaluative step taken during a woman’s career, including funding, publication, and even the rate of citation of her work upon which the h-index depends. Therefore, to use the identical criterion for women and minorities will require them to achieve the same outcomes as white men, despite repeated biased judgments along the way.” — Underrepresentation of Women and Minority Awardees in Geoscience Societies
“Results also indicate that female graduate students are less likely than male graduate students to publish, a gap that remains in the years following graduate school. Finally, results indicate that U.S. citizen minority students exhibit lower levels of publishing success compared with non-minority students during graduate school, but that this gap that disappears within the first few years after graduate school.” — Citizenship, Gender, and Racial Differences in the Publishing Success Of Graduate Students and Young Academics
(Think about what that last one means for women/gender minorities of color.)
“Most people would agree that evaluating and rewarding employees based on merit is fair and legitimate. However, merit-based practices may not naturally increase workplace equity. In fact, by adopting a purely meritocratic environment, where gender and social disadvantages are not acknowledged, biases and stereotypes may actually be accentuated.” — The Paradox of Meritocracy in Organizations
“Despite being a widely used measure of academic impact, the h-index tells you little more than the square root of a scientist’s overall citation count.” — Friends Don’t Let Friends Do H
“However, the h-index is only meaningful if it contains information you couldn’t learn from just counting citations and papers . . . Ranking scientists by their citation count seems simplistic — if we were happy doing that there would have no real need to invent the h-index. However, if the h-index is pretty close to the square root of the citation count, comparing h-indices is not too different from comparing citation counts and we are back to square one.” — Measurer, Measure Yourself.
“
