Why Is It So?
There are many economics. There are many ways of doing economics; many kinds of questions we seek answers to; many ways to try and find answers; many ways we may think we have gained some kind of “knowledge” that justifies our disciplinary activity. With all of this in mind, I find critiques of economics and calls to reform its teaching often suffer from serious vagueness. What part(s) of economics are we trying to reform, and to what end? What do we want our economics to do, and how will we know when we’ve reformed economics effectively?
(Don’t get me wrong. I can find plenty of things to criticise economics for, and I think more pluralism and real-world relevance in standard economics curricula would be significant improvements for their own sake. But these are means, not ends. We need to be able to identify particular failings and criteria by which we think improvements have been achieved.)
Forecasting and prediction are going to be considered differently here; i.e. I’ll treat them as different things. Forecasting will be considered to concern particular events; prediction will be considered to concern conditional (if-then) statements drawn from particular models. This is not a consistently recognised terminological distinction. Nate Silver uses prediction in his book explicitly to mean what I mean by forecast, for example. So this labelling scheme is particular to me. But the distinction is one that most practitioners and scholars will recognise.
Of course, it has to be noted that the distinction is not always clear-cut. Some analysts, often in the role of consultants, use models that have “predictive” properties (adjust a variable or two, and observe the changes that emerge as model outputs), and draw quite specific conclusions about what the likely magnitudes and timing of the impacts will be.
The heart of the matter
Until only recently, the Yale economics department website featured an item in its news section concerning public comments made by long-time Yale professor John Geanakoplos, from way back in the time of the Clinton presidency, about debt. He noted, in simple English, that increases in debt had been occurring without matching increases in collateral. As indebtedness increased, and debts were repackaged and then sold on, perhaps several times, the underlying collateral stayed the same, and hence the risk associated with default increased, as now there were multiple agents in a chain if indebtedness who would all be seeking claim on an existing stock of collateral.
Geanakoplos made no claim as to when a crash would happen; he simply warned of a build-up of conditions that increased the risk of a large crash happening sometime. Debt was increasing, limited collateral was being stretched further, and the consequences of a given default were becoming potentially more severe. Risks were building, and Geanokoplos pointed to them. It turned out to be years before the crash he was expecting actually arrived.
So this doesn’t really count as a forecast; more a “prediction” of the sort that says “if this current trend towards greater indebtedness continues, the effects will be bad when they happen.” It has no particular precision, although it suggests that economists who knew what to look out for could see signs of increasing fragility in the economic system. As Noah Smith has pointed out, sorting out who forecast (he uses the term prediction, but using my terminology he’s referring to forecasts) the 2008 GFC is tricky. Not least, what thing(s) about the crisis did someone forecast (particular event; timing; duration; severity; particular characteristics or causes), what degree of confidence was a forecast made with, what credibility might be attached to it, and on it goes.
So, if Geanokoplos’ “prediction” does not count as a precise forecast, or a robust empirical claim based on analysis from a formal model, what is it? Is it in any sense useful knowledge? Arguably, yes it is. Remember, he is in many ways a very conventional microeconomist (particularly, a general equilibrium and finance theorist) at an elite university; there is nothing especially heretical or extraordinary about his methods. But it’s not clear that many other leading economists were worried about debt as much as he was at that time — particularly, not just the amount of debt, but the balance of debt relative to collateral underpinning it — to the point of sounding warnings about it. The potential value of his calling attention to the risks of high leverage could lead to a policy response; likely something other than a standard stabilisation-policy response involving adjustment of interest rates or tax rates.
Anyway: something we don’t do enough of in economics circles as we argue and hand-wring about the state of economics and/or the need to reform (or revolutionise) how we do it and how we teach it is to compare what constitutes useful knowledge (understanding, insight, explanatory power, predictive success, forecasting accuracy, or whatever you want to include) in our discipline with what constitutes useful knowledge in other disciplines. So, for most of the rest of this discussion, I want to do that, to make a few points.
In particular, I’m going to focus on “health and medicine”, which covers the work done by everyone from specialist medical researchers, to public health academics and practitioners including epidemiologists, to drug/medicine companies, all the way to GPs who see patients daily.
To start with, I’m going to make some claims that work in health and medicine generally is science-based (grounded in chemistry and biology), it has proceeded alongside improvements in medical technology, and that in aggregate, the health/medical professions have improved human longevity and human welfare through science-based practice. Many conditions, diseases and injuries are now treatable that did not use to be. We know more, medically, about many things we did not know so much about before. The meta-discipline of health and medicine seems to be one we can safely describe as successful, applicable, and improving over time.
What does this set of knowledge mean at the level of the GP? A patient attending a GP may expect general expert information on health and lifestyle, including exercise and diet. A GP would also expect to spend a good deal of time dealing with the diagnosis and treatment of particular maladies, and to know when to refer patients to particular specialists. (This knowledge isn’t exact; the expression “second opinion” is often associated with the practice of seeking advice from another doctor, not least to see if it matches the advice from the original doctor.) Similarly in a hospital, diagnosis and treatment of particular conditions are common actions required of doctors, with them often being more severe than would be the case for a treating GP.
What characterises the knowledge and expertise here? It varies. It’s quite possible that a doctor might be able to diagnose and treat something effectively without having a clear understanding of causality. That is, they might say of a condition that “We can’t say why it happens, but we know what to do when it does.” (When my partner was last in hospital, that, paraphrasing, is exactly what they did tell her.)
Doctors also can have very well researched population-level understanding of links between actions and outcomes that can be expressed probabilistically for individuals: “If you smoke you’ll increase your likelihood of cancer.” They could add probabilities to that statement (“by x%”). Yet precise forecasts for persons are close to impossible as variation in outcomes between individuals is extreme (“I’ve smoked for 50 years and I’m still healthy”). This is the case even with a sophisticated understanding of cause-and-effect of how smoking causes health problems, right down to the key mechanism of damage done to DNA; medical professionals cannot accurately forecast which individuals will succumb to which form of cancer and when. But predictions of outcomes for populations can be made with considerable precision.
To develop new treatments requires both knowledge of underlying science (e.g. chemistry), as well as the use of experimental methods (trials) to test them. (To investigate effects of lifestyle on health and longevity is another matter again, which we return to below.) There are some widely used public health measures that are controversial for “scientific reasons” — e.g. water supply fluoridation, vaccination of children — but the controversies are largely outside the scientific community, with little dissent within.
We enter a more challenging area when talking about general (but expert) knowledge regarding things like diet, health, exercise, longevity, and obesity. Expert opinions — and advice to the populace — have varied widely when it comes to oils, fats, sugar, carbohydrates, salt, chocolate, and red wine. The idea of a “food pyramid”, suggesting pictorially how to divide one’s diet up into “food groups”, goes back several decades, and in terms of ubiquity could be said to be spectacularly successful. Of course, the problem is that the food pyramid proposed originally by the USDA has been controversial, both for its weak scientific foundations and for the degree to which corporate interests were able to influence its design. Of course — like economics — food and health science is going to be subject to commercial influence, but it’s not like the things being presented as expert opinion were scientifically implausible. At the core of the matter, we simply didn’t know enough.
Just take a look at the Conversation, the academia-meets-journalism site that publishes stories from the frontiers of research and policy analysis. In its Health and Medicine section there are recent stories that directly aim to challenge, if not overturn, conventional wisdom; on antibiotic use, on using painkillers to treat back pain, and on sleep patterns. We (well, medical researchers and epidemiologists) are still generating insight into what are still contested areas concerning the basics of lifestyle: what we should eat, how we should exercise, how long should we sleep for? That’s not to say that this denotes some kind of systemic inadequacy or even academic scandal in a discipline that’s vital for human well-being. It might be that there’s simply lots of messy data and it is hard to extract accurate signals from copious noise. Even questions such as whether it’s sensible to actively screen for potential conditions such as prostate cancer are contested.
One would expect that over time, our knowledge and understanding will improve and we’ll learn more about these questions. What I mean is that, despite some major gaps in our knowledge, there is a realistic prospect that the research community is going to make serious progress in closing those gaps. Gathering and sorting through large datasets is difficult, costly, and time-consuming, particularly given some datasets need years or decades to compile properly. Different kinds of data (e.g. from controlled trials; observational data; self-reported data) will be needed to validate findings, but this is simply a feature of how difficult it is to answer questions about “marginal” (but not necessarily small) effects of lifestyle changes on life outcomes. This applies to questions about particular exercise regimes as it does to diet advice and to whether regular consumption of red wine reduces or increases cancer risks.
(Gaps in knowledge about curing cancers, as opposed to knowing what causes cancers, is much more amenable to laboratory analysis and experimentation; the inherent difficulties in achieving breakthroughs are much more to do with the complexities of cancers themselves than of having to work with large and messy datasets across populations where causality is difficult to ascertain from many possible correlations.)
“Science-y” approaches have been problematic in areas like nutrition in an era where processed foods have come to dominate many people’s diets. In The Omnivore’s Dilemma, Michael Pollan critiqued the reductionist approach of developing manufactured “food-like substances” that have been prepared as nutrient delivery systems with a focus on a particular nutrient that marketing has convinced us we need more of. (The desired nutrient itself may change over time.)
The point of this is not to point the finger at the health research community and highlight its shortcomings. It’s, firstly, to make the point that knowing things is difficult, and that the health and medical research community are pretty good at figuring things out, even as they face challenges that will take time and effort and persistence to solve. I’ve barely touched on the political and commercial pressures applied to how R&D resources are allocated; particular diseases are prioritised, and particular problems are targeted, without it being made clear that those things prioritised present the greatest potential gain in human welfare per research dollar invested. But it’s not like this field is unique in facing such pressures.
Secondly, it’s to demonstrate practically the idea of “multiple ways of knowing” in a meta-discipline that’s crucial for human welfare.
- Precise forecasts of events are generally difficult to make and unreliable.
- Aggregate population-level predictions can be more reliably made.
- Being able to identify risk factors is critical — which is to say, having some understanding of the relevant cause-and-effect linkages — even if that does not enable you to make accurate forecasts.
- Diagnosis (from circumstances and symptoms) and identification of appropriate treatments represent valuable knowledge even when cause-and-effect regarding the condition itself is poorly known.
In short, “explanatory power”, “predictive power”, and “forecasting accuracy” can be thought of as different aspects of knowledge, and they may well be linked, but it’s possible to have them in different combinations. Being able to explain risk factors in detail may enable generalised predictions across populations to be made, but not forecasts about what will happen to an individual. Predictions about the efficacy of a treatment may be made confidently without a causal understanding of why the condition manifested itself when and where it did. Randomness in an individual and heterogeneity across populations limit our ability to treat our knowledge as deterministic, but it seems like a harsh judgement indeed to say we don’t have useful and verifiable knowledge in health and medicine.
Changing disciplinary focus, geologists and and seismologists have struggled to accurately forecast earthquakes, even while their knowledge of why earthquakes happen and where risks are greater (i.e. explanatory power) has increased. In the field of education, something as fundamental as how best to teach children to read is fraught with debate, suggesting that our understanding of the process of learning (explanatory power) is less than stellar.
Economics, even in the best of all possible worlds with an optimally evolving economics discipline, would face similar challenges. Chris Dillow notes the distinction between forecasting and explaining, and cautions us against judging economics based on forecasting accuracy, since forecasting might simply be too difficult to get right regularly in a complex world. This notion needs to be discussed openly as we engage in discussions and conferences and blog arguments about “what’s wrong with economics”. Dillow quotes Jon Elster that “Sometimes we can explain without being able to predict, and sometimes predict without being able to explain.” Exactly.
(In a microeconomic context, a discussion of the fate of Australia’s wool reserve price scheme, given the setting of a vastly inflated price by the scheme’s managers, contained this comment from an agricultural economist: “The probability of [the scheme’s] collapse rapidly converged on 1.0; only its timing remained uncertain.” Accurate knowledge in terms of precise forecasting is limited, but useful knowledge of a probabilistic sort is possible, and valuable.)
If that’s the case though, what can economics do? Can we diagnose? Can we treat? Can we identify risk factors, even while not being able to put precise dates on future crashes? Can we identify precautionary policy steps that could be taken given a set of identified risk factors?
As an example — a prominent, but not definitive, one — Paul Krugman is of the view that the tools and models we have in economics are sufficient both to explain how it happened, and what to expect after it did (and how best to deal with it). If he’s right, this knowledge would surely have provided insights on precautionary (if possible, preventative) actions to take to stave off a crash, and remedial actions to introduce once a crash happened. Krugman is, I think, pretty clear in his view that we had the tools to manage the economy after the crash effectively, and that our failure to do so was political rather than technical. That political failure was supported to some extent by particular economists who pushed back against stimulus actions, citing structural explanations for unemployment and fears of inflation from Fed stimulus.
(If you go back to my previous Medium post on the Unholy Trinity, the Krugman position on remedy might be summarised as that the discipline was appropriately equipped, but individual economists acted to derail responses consistent with disciplinary knowledge.)
I’m less clear on Krugman’s position as to what could have been done prior to the crash, even though he says the discipline had what was needed in terms of explanatory power. If irrational exuberance that increased leverage across the economy was the driving force, a view that seems consistent with Geanakoplos, might there have been some financial/regulatory intervention that might have eased conditions, reducing either the chances or the eventual severity of a crash? I’m not sure what Krugman thinks here, but it does seem like few economists were paying much attention to these issues and sounding warnings that might have been heeded.
(Again, with regard to the Unholy Trinity, if this represents a fair picture, it is more a failure of the profession than an inadequacy of the discipline.)
So, about economics
There are arguments to be had about modelling strategies in economics, about economists’ ability or otherwise to forecast, about theoretical modelling “versus” empirical work, and about whether mainsream approaches marginalise alternative (“heterodox”) approaches. Those arguments will happen in other posts, not this one, which is already bursting at the seams.
This post is written to address concerns I have about claims that we need to reform economics research and education, and perhaps to revolutionise it, to overthrow the orthodoxy, to abandon much of what has gone before and to embrace radical new approaches. I’m pretty sure that economics could use some reform; I am less convinced about the benefits of revolution, but I am also sure that I am not the person to judge such claims.
What I do hold to quite firmly is that to judge what is good or not good about economics — when is it useful, and in what way? — we need to spend time away from debates about modelling strategies, methodological nitpickery, and quibbles over assumptions. It’s not that those things don’t matter — they do — but if we work through how economics is useful, or how it can/could be useful, and how that usefulness compares to the usefulness and limitations of other disciplines, we start to get a sense of how we developed that usefulness. We also can get a clearer vision of what’s impeded us in developing usefulness.
And that way, when we get involved in reforming the discipline and its teaching and research, we’ve got a decent chance of throwing away the bathwater while keeping the baby safe.
Footnotes, afterthoughts and further readings
- The title of this piece comes from the delightfully quirky scientist Julius Sumner Miller who presented a series with that title on television over a number of years.
- The John Geanokoplos quote I stumbled upon some years back had been on the Yale site for ages, and I only realised it was gone (in a site redesign) when I went looking for it to quote it for this piece. However, he ended up writing quite a bit about debt and leverage, not least this piece which was part apologia, part rationalisation of his time working in private finance. [Also, this.]
- I don’t mean to suggest that Geanokoplos was the only economist concerned about debt prior to the GFC. I only used him as an example of an economist seeing danger signs and warning of them, in a very simple and plausible way, without formal modelling or published empirical results (even though he did write formal papers such as the one linked above). Dean Baker obviously paid attention to these things too. But the question remains, how many economists were looking out at the world and seeing “red flags” and speaking out? Of those who did, were they ignored, and why? Of those who didn’t, why didn’t they? (Daniel Davies has said that most economists didn’t know what a CDO was prior to the crash.)
- The Queen’s comments have been variously quoted, but “Why didn’t anybody notice it?” is the commonly reported wording. This seems to me more like “Why didn’t anybody see this coming?” as distinct from “Why did anybody not forecast this accurately?” Seeing something coming sounds more to me like diagnosing increasing risks, noting that these increase the likelihood of a crash occurring within some time period, and taking some anticipatory action. The written response seems in that spirit: that smart people weren’t paying attention to what they should have been paying attention to, assuming that risks remained manageable.
- Pedro Serodio tweeted me to remind me of the court-case that proceeded against several Italian seismologists who were charged, convicted, then cleared of manslaughter for failing to provide warnings of an earthquake that resulted in loss of life.