New Working Paper Set: Interpreting Recent Research on the Effects of Minimum Wage Increases Enacted During the Great Recession
Jeffrey Clemens, UC San Diego
The Minimum Wage and the Great Recession: A Response to Zipperer and Recapitulation of the Evidence
Additional Evidence and Replication Code for Analyzing the Effects of Minimum Wage Increases Enacted During the Great Recession (with Michael Wither) Note: Replication materials are available from Clemens upon request here.
Introduction
Minimum wage research regularly confronts policy makers with conflicting claims. This research summary discusses a series of recent analyses of minimum wage changes that were enacted during the Great Recession. Within that setting, I discuss why alternative research strategies generate divergent estimates. I emphasize that evaluating divergent estimates requires understanding each strategy’s underlying assumptions. This places a premium on the transparency with which research is presented.
Informative research requires transparency along two key dimensions. First, it must be clear regarding its underlying assumptions. Second, it needs to inform readers of the threats those assumptions face and of the potential size of the biases those threats entail.
A Summary of Clemens and Wither’s (2014) Analysis of the Effects of Minimum Wage Increases Enacted During the Great Recession
In a 2014 working paper, Michael Wither and I concluded that minimum wage increases contributed meaningfully to the dramatic declines in employment among low-skilled individuals during the Great Recession. Our analysis of these minimum wage changes makes use of the fact that the federal minimum wage’s rise from $5.15 to $7.25 was differentially binding across states. The following series of facts underlies our conclusions.
Fact 1: Employment declined modestly more among low-skilled individuals in states that were fully bound by this period’s federal minimum wage increase than in those that were partially bound.
Our first facts involve direct comparisons of employment changes among individuals in low-skilled population groups in “fully bound” states relative to “partially bound” states. On average, the “fully bound” states were states in which the increase in the federal minimum wage led the states’ minimum wage rates to rise by precisely or nearly $2.10. The “partially bound” states were states in which, on average, the minimum wage rose by roughly $1.40. Our initial comparisons of these states reveal that employment declined modestly more among low-skilled individuals in fully bound states (the “treatment” group) than among low-skilled individuals in partially bound states (the “control” group).
The differential employment declines we observe are largest when we are able to use relatively detailed information to identify low-skilled individuals. This is particularly true when we analyze the Survey of Income and Program Participation (SIPP), in which we are able to identify low-skilled individuals on the basis of their wage histories in addition to coarse traits like age and education. Because wage histories can be constructed in a variety of ways, we have supplemented our earlier analysis with analyses using several approaches (Clemens and Wither, 2017). This supplemental analysis includes discussion of alternative approaches’ strengths and weaknesses.
Fact 2: The forces underlying the Great Recession were less severe in the fully bound states than in partially bound states, as seen in aggregate income, labor market, and housing market data.
As in any empirical study, we arrive at our “preferred” estimates by accounting for potential sources of bias to the “direct” (or “unadjusted”) comparisons discussed above. That is, our preferred estimates incorporate adjustments for factors other than minimum wage changes that may have differentially affected employment among low-skilled individuals in the “treatment” group relative to the “control” group. The direct comparisons themselves can thus be described as the first essential input to our study’s findings. The second essential input, to which I turn below, involves evidence on the potential biases for which we adjust.
Direct comparisons may be biased due to differences in the economic developments to which a study’s “treatment” and “control” groups are exposed. In our setting, this primarily involves variations in the severity of the forces underlying the Great Recession. We thus investigate whether economic indicators provide evidence that the forces underlying the Great Recession were stronger or weaker in the treatment group relative to the control group. The comprehensiveness of the set of indicators on which we provide evidence is essential for evaluating whether we have omitted potentially important factors. We thus present evidence on a broad set of economic indicators. These include states’ aggregate economic output, multiple labor market indicators, and indicators of the performance of the housing market. Supplemental analysis in Clemens (2017a) provides additional evidence on construction employment and overall construction output.
Macroeconomic and housing market indicators make clear that the forces underlying the Great Recession were more severe in the states that comprise the control group than in the states that comprise the treatment group. The direction of this imbalance will be quickly appreciated by readers who are familiar with the geography of the Great Recession. The federal minimum wage was least binding in the Northeast, on the West Coast, and in additional states including both Florida and Arizona. On average, these states and regions experienced much more severe housing crises than the predominantly Southern and Midwestern states that were most strongly bound by the increase in the federal minimum wage.
Fact 3: When comparing states with similarly severe housing declines, low-skilled individuals’ employment declined significantly more in fully bound states than in partially bound states.
In this setting, the ideal thought experiment for estimating the minimum wage’s effects involves comparing changes in low-skilled groups’ employment across fully and partially bound states that experienced housing crises of similar severity. Clemens and Wither’s original and subsequent analyses consider several approaches to implementing this thought experiment. Each approach can, at day’s end, be described as an attempt to adjust the “direct” comparisons discussed above for observable differences in the severity of the Great Recession. Importantly, our adjustments are in line with other researchers’ estimates of the housing decline’s direct effects on employment. The analysis leads us to the conclusion that low-skilled groups’ employment fell significantly more in “fully bound” states (relative to “partially bound” states) than one would have predicted on the basis of their macroeconomic conditions.
Fact 4: The previous fact’s pattern is unique to the segment of the labor market that is targeted by the minimum wage.
Our final set of facts involves employment among middle- and high-skilled individuals for whom the minimum wage can at most have indirect effects. These groups’ employment provides a check for the presence of divergent conditions in “treatment” and “control” states’ labor markets. If moderate minimum wage changes predict employment changes among high-skilled groups, we would worry that our estimates were confounded by broader labor market developments that affected employment among individuals of all skill groups. The absence of evidence for such developments is one of the key facts underlying our reading of the evidence.
I now turn to a recent analysis that challenges our conclusions.
A Summary of Zipperer’s Comment on the Original Clemens and Wither Analysis
In a comment on our analysis, Zipperer (2016) arrives at very different conclusions. Because our analyses use the same data, differences in our estimates must stem from differences in the sets of comparisons on which they ultimately rely. Zipperer’s comment thus creates an opportunity to explore the strength of the assumptions underlying alternative approaches to estimating the minimum wage’s effects.
Zipperer’s primary line of analysis adopts an increasingly widespread assumption that “geographically proximate” comparisons are preferable to broader comparisons. In brief, his primary line of analysis restricts attention to comparisons between “fully” and “partially” bound states that lie within the same census region or, alternatively, the same census division.
“Geographically proximate” comparisons have recently become quite popular in applied econometric analyses. It is thus important to emphasize that within-region comparisons can, for reasons that have long been recognized, be subject to more, less, or similarly severe biases as comparisons that extend both across and within regions. The fact that within-region comparisons have become popular does not alter this point. On a setting-by-setting basis, the applicability of each estimation framework’s underlying assumptions still needs to be studied. In the following section, I assess the assumptions at work and discuss evidence on what drives the difference between Zipperer’s estimates and our original estimates.
What Underlies the Difference between Zipperer’s Estimates and Clemens and Wither’s Estimates
Two key factors underlie the difference between Zipperer’s estimates and Clemens and Wither’s estimates. The first key factor is that Zipperer’s estimation strategies significantly reduce the policy variation underlying the estimates. The analysis in Clemens (2017a) shows that, on average across Zipperer’s regressions, the treatment’s “dosage” (i.e., the size of the minimum wage change underlying the estimates) is reduced by more than one third. I show that this “dosage reduction effect” accounts for nearly half of the difference between Zipperer’s estimates and the estimates from our original analysis.
This first factor’s relevance can be intuitively conveyed through a simple analogy: just as two Aspirin tend to provide more pain relief than one, a large minimum wage increase will tend to have a larger employment effect than a small minimum wage increase. In the jargon of applied econometrics, Zipperer’s regressions significantly “attenuate” the size of the minimum wage changes underlying the employment effects he estimates. Intuitively, estimates must be scaled to account for differences in the “treatment dosage” before they are directly comparable. Because Zipperer does not report this set of facts, his accounting of the differences between his estimates and the Clemens and Wither estimates is incomplete.
A subtler question is whether within-region comparisons are exposed to smaller or larger biases than comparisons that extend both across and within regions. The analysis in Clemens (2017a) devotes considerable attention to this question. The short answer is that within-region comparisons are exposed to larger biases. The mix of “treatment” and “control” states in the South helps to illustrate why. In the South, Florida accounts for the majority of the sample from the partially bound “control” group. This is problematic because Florida experienced a housing crisis far more severe than those of its regional neighbors. I show in Clemens (2017a) that, adding up across all regions, the “within region” comparisons on which Zipperer relies are prone to considerable bias.
The biases associated with within-region variations manifest themselves in multiple ways. The first is that Zipperer’s estimates are quite sensitive to excluding states with extreme housing declines from the sample, while the Clemens and Wither estimates are not. This reveals that his results are driven in part by the weight they place on comparisons involving states that experienced relatively severe crises. The second involves employment among relatively high-skilled individuals. As shown in Clemens (2017a), Zipperer has generated specifications in which modest minimum wage changes predict considerable employment gains among high-skilled individuals for whom the minimum wage has no direct effect. This provides evidence that his estimates are driven by comparisons between states in which the “control” group experienced far worse labor market shocks than the “treatment” group. Indeed, this is precisely what is implied by the aggregate income, house price, and construction output data discussed above.
Zipperer’s estimates thus differ from Clemens and Wither’s estimates for two primary reasons. First, his specifications substantially reduce the effective “treatment dosage.” Second, his specifications shift weight to comparisons that are prone to considerable upward bias.
Additional Methodological Considerations
The analysis summarized above diagnoses the difference between Zipperer’s estimates and the original Clemens and Wither estimates. I show that the premise underlying Zipperer’s preference for within-region comparisons is difficult to reconcile with evidence from aggregate income, labor market indicators, and housing market indicators. The indicators reveal that the within-region comparisons on which his estimators rely are prone to considerable upward bias. More specifically, they are exposed to greater biases than comparisons that extend both across and within regions.
There is a further point worth emphasizing. Zipperer’s focus on within-region comparisons is one of many restrictions one could imagine making to the full set of available comparisons. It is thus worth emphasizing that his exercise generates less information than one might initially think. An analysis restricted to a subset of the available comparisons informs us of a subset of the available evidence.
Restrictions of the sort Zipperer imposes should be examined with standard data mining concerns in mind. The key point is that the decision to discard comparisons needs to be justified by evidence that imposing restrictions genuinely improves the prospects for generating unbiased estimates. In this instance, aggregate income, labor market indicators, and housing market indicators provide evidence that the restriction has, if anything, exacerbated sources of bias rather than reduced them.
Falsification Tests
In an additional piece of analysis, Zipperer reports evidence from an exercise that he describes as a “falsification test.” Like within-region estimators, falsification tests have become increasingly popular in applied econometric work. The purpose of such exercises is to pose a “test” that has bearing on the validity of some underlying estimation framework. As with any empirical exercise, the information content of falsification tests depends on the credibility of the assumptions on which they rely. Interested readers can find an analysis of the key premise of Zipperer’s falsification test in Clemens (2017b).
The premise of Zipperer’s falsification test is quite specific. In brief, the test’s validity requires that minimum wage changes were as good as randomly assigned within the South. As discussed above, this is quite obviously not the case; Florida, which accounts for a majority of the South region’s “control” group, experienced a far more severe housing crisis than its regional neighbors. In addition to showing that the test is biased, the analysis in Clemens (2017b) shows that it is far less statistically informative than implied by the confidence intervals Zipperer reports. Finally, the falsification test rejects one of the primary specifications Zipperer advances as being preferred to the Clemens and Wither baseline. In this respect, the analysis thus lacks internal consistency.
Conclusion
This research brief summarizes a series of empirical analyses of the minimum wage changes enacted during the Great Recession. The evidence supports the conclusion that this period’s minimum wage changes contributed meaningfully to declines in low-skilled groups’ employment.
It is certainly possible that there are biases for which neither our original nor subsequent analyses have managed to account. Analyses of observational data are, due to their inherent limitations, unable exhaust such possibilities. The analysis is sufficiently comprehensive, however, that lingering sources of bias would have to involve economic forces with remarkably nuanced effects. Lingering biases would have to involve factors that affected low-skilled groups’ employment while leaving no detectable trace in statistics including aggregate income, construction output, and house prices. Further, they would have to involve factors that had no effect on employment in other segments of the labor market. That is, they must be factors that uniquely affected the segment of the labor market that is targeted by the minimum wage.
These analyses contribute to our understanding of the minimum wage’s effects during an important, but distinctive, historical episode. The Great Recession was a period during which labor demand was depressed, inflation was moderate, and productivity growth was slow. These are precisely the conditions under which a minimum wage increase’s bite can be both deep and long lasting. The effects of more recent minimum wage increases ought to be similarly analyzed and understood with reference to the labor market conditions into which they are being implemented.
References
Clemens, J. 2017a. “The Minimum Wage and the Great Recession: A Response to Zipperer and Recapitulation of the Evidence,” Discussion paper.
Clemens, J. 2017b. “Pitfalls in the Development of Falsification Tests: An Illustration from the Recent Minimum Wage Literature,” Discussion paper.
Clemens, J., and M. Wither. 2014. “The Minimum Wage and the Great Recession: Evidence of Effects on the Employment and Income Trajectories of Low-Skilled Workers,” NBER Working Paper 20724. See the link below for the latest version: http://econweb.ucsd.edu/~j1clemens/pdfs/ClemensWitherMinimumWageGreatRecession.pdf
Clemens, J. and M. Wither. 2017. “Additional Evidence and Replication Code for Analyzing the Effects of Minimum Wage Increases Enacted During the Great Recession,” Discussion paper.
Zipperer, B. 2016. “Did the minimum wage or the Great Recession reduce low-wage employment? Comments on Clemens and Wither (2016),” Discussion paper.
To learn more about ESSPRI, visit esspri.uci.edu.
