Part 1: The Big Problem, And The Basic Questions.
“Mass incarceration” usually refers to the historically high U.S. incarceration rate, which is the share of the population in state and federal prison at any given time. Starting in 1980 or so, incarceration rates had huge growth after decades of relative consistency, which is often illustrated with some version of the following graph:
Graphing the history of incarceration rates is probably one of the most successful, effective, and iconic examples of data visualization. It makes a clear, simple, and dramatic point, and it’s been repeated countless times.
But there’s a big problem. The incarceration rate — who’s in prison right now— follows from two basic questions, which those famous graphs (and phrases like “the prison population”) mash together:
- Who did we send to prison in the first place, and for what? How much has “mass incarceration” meant expanded incarceration? Which people are going to prison who’d never have gone under other policies? There’s a moral version of the question, too: What should we send people to prison for?
- How long were they in prison for? How much has “mass incarceration” meant deepened incarceration? Which people would have gone to prison anyway but are spending more time incarcerated, either serving longer sentences or cycling through the system over and over again? Again, there’s a moral version of the question: How long should people be in prison for?
Both of those questions are important, but they’re distinct topics, at right angles to each other analytically and morally: “should drug offenders go to prison at all?” is pretty different from “how long should convicted murderers serve in prison?”. The next time you read an article about incarceration in the popular press or on social media, though, try to see if it’s even aware of the distinction. (For example, “reducing the prison population” is a pretty meaningless phrase — does it mean having fewer people go to prison or having people go to prison for less time? One reason you don’t understand mass incarceration: the phrase “the prison population”.)
It’d be hard enough to understand how those two questions interact if we knew how to answer both of them, but no one knows how to answer either one. There’s been a lot of impressive, interesting work trying to answer who goes to prison for how long (John Pfaff has been a big influence on this piece) and even the best of it is hampered by limited or unreliable data and difficult inferences, while obviously there isn’t much consensus on who should.
That’s a problem, despite all that great work by dedicated scholars, activists, and writers. Once you realize no one knows these answers, then you’ll realize why you rarely see people talk about exactly who is going to prison now who wouldn’t have gone to prison in 1977. It can be frustrating to read article after article that confidently says, for example, that the United States has obviously put far too many people in prison for far too long but somehow never quite gets around to specific claims about how many people actually went to prison and how many of them shouldn’t have or should have but for less time. At least admit that you don’t know!
What we call “mass incarceration” is a complicated system of policies and choices affecting millions of people directly and indirectly. The consensus and debate are shifting right now; bills and reforms are being drafted and passed. Everyone’s talking about mass incarceration, but no one, myself included, really understands it. We won’t understand it until we move away from focusing on incarceration rates and “the prison population” as such — move away from focusing on the easiest things to count, just because we can count them — and move towards focusing on who goes to prison and for how long.
Part 2: We Don’t Even Know How Many People Went To Prison At All…
Let’s start with what should be the easiest question, the first part of the first one.
Quick: How many people really were sent to state or federal prison in the United States since 1978? Don’t be surprised if you don’t know, since as far as I can tell no one else does either.
If you care about incarceration, then that should trouble you. This is, after all, the set of human beings that we’re talking about when we talk about mass incarceration. If we really understood mass incarceration then we would know a great deal about them. We would know how many were African-American or white, how many served one, two, or ten terms in total, how many had different kinds of criminal histories prior to incarceration, how many only went to prison on drug offenses, how many went to prison on drug offenses then went back on violent offenses, and so on (and how all of those things changed over time and between different states and places).
But we don’t know any of that. We don’t even know how to count them. And if we don’t know how to count how many people have been to prison, then we don’t really know how mass incarceration affected real human beings. [1]
We have some idea of how many prison terms there have been. The Bureau of Justice Statistics (BJS) lists about 12.7 million “new court commitments” to state and federal prison since 1978 in their National Prisoner Statistics (NPS) data, which is my main source here. Of course, that doesn’t mean that 12.7 million different people were sent to prison between 1978 and 2014, since some people had multiple prison terms.
The typical approach to trying to count distinct individuals relies on self-reported incarceration histories from semi-regular surveys of inmates, like the Survey Of Inmates In State Correctional Facilities (SISCF). That’s the source of claims you may have heard like “one in three black men go to prison at some point”, but does it seem like the best possible approach? Anyway, there hasn’t been such a survey since 2004.
Still, we can try to estimate how many people have been to prison. Based on a few sources, I’d say that the number of distinct people is probably between one-half and three-fourths of the number of new court commitments [2]. That would more or less mean between 6.8 million and 9.5 million people were sent to state and federal prison since 1978, maybe. If I had to guess then I might say around 7.5 million people.
I’m not confident in that answer. To really know how many people were sent to prison would require reliable data at the individual level back to 1978 if not before, for every state in the country along with the federal system, that followed the same people across different prison terms within and between systems. I’m sure that’s all in some database somewhere, but there simply isn’t any data set like that publicly available. The closest, the National Corrections Reporting Program (NCRP), has a lot of problems: It’s only accessible to certain kinds of researchers, it isn’t considered reliable for more than a handful of states, and it only links people across prison terms back to 2000 or so, and then only within states rather than across states or with the federal system. (I think. I’m not one of the researchers it’s accessible to.)
Without a longitudinal, comprehensive record of criminal histories at the individual level, or at least a sense of what one would look like, we won’t really understand mass incarceration. Until then, all we can do is try to approximate it in our minds.
…So We Certainly Don’t Know How Many People Went To Prison “Because Of Mass Incarceration”….
Let’s say I’m right that around 7.5 million people were sent to state and federal prison since 1978.
Here’s another question that should be simple: How many of them wouldn’t have gone to prison without “mass incarceration” policies — and what does that mean, anyway? It’s clear that new prison terms did increase after 1978, so presumably a lot fewer people would have gone to prison without whatever it was that changed, but how many fewer people? More like two million or more like six million?
Nobody knows, but we can make up some numbers, and maybe that’s a start. For example we can consider a somewhat arbitrary hypothetical: What if new prison terms (new court commitments) had stayed constant per capita since 1978, when the rate was 57 in 100,000? Under that hypothetical, there would have been about 5.6 million new prison terms since 1978, not 12.7 million.
I’ve illustrated this in the following chart, which has the incarceration rate since 1978 in black, the rate of new prison terms in red, and the hypothetical of constant new terms per capita in blue.
Using the same guess as before about going from new court commitments into distinct individuals, those 5.6 million new court commitments under the hypothetical scenario would translate to 2.8 to 4.2 million distinct people sent to prison from 1978 to 2014. So, maybe 3.5 million people would have gone to prison under the hypothetical, and maybe 7.5 million people really did go to prison, so maybe we can guess that about 4 million people went to prison “because of mass incarceration” [3].
Perhaps a lot of those 4 million people would have had some level of criminal history either way: arrests, juvenile incarcerations, or misdemeanor and even felony convictions that didn’t result in incarceration. (A 1996 report by John DiIulio and George Mitchell on Milwaukee prisoners found that “just one in 10 inmates had no prior adult or juvenile conviction” and that the vast majority of these were violent offenders.) Honestly, I don’t have much sense of what kinds of people are going to prison now who would never have gone to prison in 1977 but hopefully this will give you some sense of the possible scale.
Of course, that’s quite a few “maybes” and a pretty arbitrary hypothetical. For example, it’s possible that new court commitments started rising a few years before the incarceration rate, so you could pick a smaller “constant rate” for your hypothetical, although maybe that’s implausible too given the crime rate at the time.
You could also argue that a better hypothetical would have new prison terms dropping with the crime rate. However, as my first chart shows, the arrest rate did drop, while new court commitments per capita stayed high and even increased, which proves…something? Pfaff thinks it shows that mass incarceration was driven by prosecutors becoming more aggressive and more willing to file charges, which is certainly possible but not necessarily the only possibility — maybe arrests fell because police started focusing more on offenders who were somehow more serious.
Again, until we have reliable state-by-state data that breaks down arrests and new court commitments in similar terms, preferably at the individual level, we won’t understand this. Every state has a different incarceration history: Some of them (like Maryland, Oregon, North Carolina, and Michigan) had relatively steady new court commitment rates, while others (like Virginia, Ohio, Arizona, Louisiana, and Illinois) record new court commitments nearly tripling or more. Does that mean that Michigan didn’t send a lot of people to prison “because of mass incarceration” but Ohio did? Why?
…Let Alone How Many People “Should” Have Gone To Prison.
Here’s a different question (unless you think the US had perfect criminal justice policies in 1980 or whenever) but maybe the most important question:
How many of those maybe-7.5-million-or-so people who went to prison “should” have gone to prison, based on whatever your ideal policies are? For that matter, how many people didn’t go to prison but “should” have, based on whatever your ideal policies are?
Unless you’re a prison abolitionist and your answer is “none”, then those are hard and subjective questions. But they’re the questions we should be trying to answer if we want to understand mass incarceration. Again, if you support decarceration, then you should be able to say “The United States sent 7.5 million people to prison, or whatever the real number is, but X million of them shouldn’t have gone in the first place, and we need to change our policies so people like those X million don’t go to prison any more”, and the same if you support incarceration. Otherwise, what is anyone talking about?
For example, there’s a consensus among a lot of people that the “drug war” was and is wrong, so perhaps they debate the analytical question “how much did the war on drugs contribute to mass incarceration?” as a way to try to approach the moral question “how many people went to prison who shouldn’t have gone to prison?”. I bet that’s why that topic attracts to much attention.
Still, it’s a hard question to answer, or even to know how to define what an answer would look like. How many people only went to prison on drug charges? If you’ve read this far, then it’ll be no surprise that no one knows.
It does seem like drug offenses and/or “other offenses” (think traffic offenses) account for well over half of the increase in new court commitments since 1978, but I don’t think we know how that maps to actual individuals. Looking at two subsets of states with more-reliable NCRP data, Pfaff found that about 20% of individuals incarcerated at some point from 2000 to 2014 were “pure” drug offenders, which would seem to suggest that any increase in such offenders accounts for less than half of the increase in total offenders.
As far as I know, we also can’t say how many people plead down to drug offenses from more serious charges that they might have been convicted of without the emphasis on the “war on drugs”, or how many people were incarcerated on drug offenses in part because they weren’t incarcerated on earlier and more serious charges. (That DiIulio/Mitchell report looked at 12 drug offenders and 7 had previous violent charges as adults.) On the other hand, we don’t know how many people were convicted of violent offenses that in some sense resulted from the “war on drugs” — that wouldn’t have happened without it.
So we really have no idea “how many people went to prison because of the war on drugs”, let alone “how many people went to prison who shouldn’t have”. If we can’t say how many people went to prison who shouldn’t have gone in the first place, then we don’t understand mass incarceration.
Part 3: We Don’t Know How Long People Were In Prison For, Or Why.
Trying to understand how long people stayed in prison and why poses the same problems as trying to understand who went to prison and why: We don’t have reliable individual-level historical data, and even if we did, it would be hard to interpret under hypothetical scenarios and counterfactuals.
Derek Neal and Armin Rick, looking at a subset of states where they considered the NCRP data to be reliable, found that at least two-thirds of prisoners admitted from 1985 to 2000 served for under two years. That doesn’t take into account people serving multiple terms because of recidivism, people re-admitted after parole violations, and so on. But they also found that offenders became somewhat more likely to serve for at least five years, mostly in the categories of “Murder and Homicide”, “Robbery”, “Other Sex Crime”, and “Aggravated Assault” [4].
Even without relying on the NCRP, or on other people’s interpretations of the NCRP, it does seem like time spent in prisons has increased for some group of offenders. Look at the last graph again and you’ll see that prison populations sometimes grew faster than new court commitments. The ratio of the incarceration rate to the admissions rate, which is something like a bad proxy measure for the average time served in a single prison term, did increase from around 2.5 to around 3.5, mostly during the 1990s.
This is easier to see if we look at that ratio directly, as in the following chart:
That would correspond to a 40% increase in average time served, which is close to the 36% increase that Pew found looking at the NCRP across states — but, again, the NCRP isn’t usually considered reliable for most states. There’s other evidence: Ashley Nellis and the Sentencing Project conducted their own survey of corrections departments (and they have my thanks for gathering new data!) and found that “in the early 1980s roughly 4% of prisoners were serving life sentences…[compared to] 10.6%” in 2012 or 2013, although I’d rather see a comparison by percentage of admissions, since prisoners serving life sentences accumulate over time.
Even so, that doesn’t establish who time served is increasing among, or whether it’s longer individual terms or increased cycling through parole violations (California was notorious for this until recently, and nationally, total admissions grew faster than new court commitments). And, once again, it doesn’t show the effect on individuals across prison terms.
Just as an example of how complicated this can be, consider Texas. Why did Texas’ incarceration rate go up so sharply around 1993, when new prison terms didn’t?
The answer, or at least a large factor: Texas, like other states, had been operating under an overcrowding lawsuit, and they built a huge expansion of the physical prison system to accommodate that growth. (Some of the seamier side of this is discussed in this Texas Monthly article by Robert Draper: “The state prison system, which before the buildup was so overcrowded that it had to turn inmates loose after only a few months behind bars, now has 146,000 prison beds but only 129,000 inmates.”) Once Texas had built more prisons, they could keep people in prison for longer, although they also did adopt a new penal code in 1993 (and revised it in 2003, which might account for some of the drop in incarceration rate since then) .
Part 4: Conclusion.
If you think that the drug war is wrong, or that longer sentences for violent offenders are right, then they’re wrong or right whether or not they can be proven to have had this or that effect on incarceration rates or whether or not they can be fully understood at all. Again, the population of people incarcerated at some point matters, and how long people spend in prison matters too.
Mass incarceration is an important topic that deeply affects millions and millions of Americans, directly and indirectly. We shouldn’t try to avoid debating it or forming opinions until we fully understand it, which is probably impossible. But we should try to remind ourselves that we don’t fully understand it, and recognize that as an imperative to try to understand more.
(Of course all of these problems are multiplied if you want to try to understand local jails, probation, etc.)
Footnotes Etc.
- Maybe it’s true that “the prison population” at any given time is roughly 40% white, 20% Hispanic, and 40% African-American, as Leah Sakala of the Prison Policy Initiative wrote, but it’s tricky: If people of color receive longer sentences on average, then they’d actually be over-represented in “the prison population” at any given time compared to the population incarcerated at some point.
- May as well show my thinking: In a 1997 report, BJS statisticians Thomas Bonczar and Allen Beck wrote that “Based on data from past BJS surveys of inmates in State prison, about 50% of all new court commitments in any year are first admissions”, citing results from 1974 to 1991. They’re the experts, but frankly that seems a little low and makes me skeptical of using inmate self-reporting for this. Florida’s annual reports give counts of the percent of admissions that had prior commitments to Florida prisons, which didn’t reach 50% until 1991 (see page 37). Since FY 1997, 54%-58% of new court commitments are listed as first admissions to Florida prisons. At a minimum that suggests there’s some variation between states and/or over time.
Pfaff’s paper on drug offenders looked at NCRP data in two sets of states tiered by reliability, and he calculated “total offenders” as 52% of “total admissions” in his “Tier A” states and 69% of “total admissions” in his “Tier B” states. Between all of that, a broad range of 50% to 75% seems like a decent guess, with the low end perhaps likelier than the high end.
3. Again, despite their expertise I’m a bit skeptical of these kinds of studies, but you can compare my amateurish guess to Bonzcar’s 2001 calculation here, or to the Becky Pettit/Bruce Western study here. Like the Neal/Rick study mentioned above, I think I heard about the Pettis/Western study via Ta-Nehisi Coates’ recent article on incarceration.
4. This, from the National Academies Press book “The Growth of
Incarceration in the United States”, is based on a study that I don’t think is public but sounds decent, depending on how you qualify “closely associated”: “The growth in imprisonment — most rapid in the 1980s, then slower in the 1990s and 2000s — is attributable largely to increases in prison admission rates and time served. Increased admission rates are closely associated with increased incarceration for drug crimes and explain much of the growth of incarceration in the 1980s, while increased time served is closely associated with incarceration for violent crimes and explains much of the growth since the 1980s.”
