Is Climate Change a Big Deal: a case study for practical data science
Short answer: probably.
A few months ago I visited a friend from college I hadn’t seen in a while. At some point our conversation drifted towards a climate change adjacent topic, at which point he dropped the bombshell that he didn’t believe in climate change. This was a bit of a shock to me — of course I operate in one of those intellectual bubbles you hear about, where I didn’t think I knew or often interacted with climate change deniers, and here I was having known one for years. It seemed particularly strange to me because this was a person I had otherwise thought of as being relatively math and science literate: he was an electrical engineering major in college and worked as an an engineer for a couple of years out of college before undergoing a career change. Not the sort of person you would think of as being incapable or unwilling to approach complicated problems or statistical reasoning.
I pressed him on it, and he ended up saying essentially that he wasn’t exactly a denier as much as he was simply apathetic. “I don’t know enough about it to say either way” he said, “I don’t think it’s a big deal.” In its way, I found that even more frustrating, a refusal to engage that cut off the possibility of even considering the issue. At the time, I saw the conversation as simply a disappointing and possibly friendship-ending cap to the visit. Now a month into an intensive data science program, I see it as an excellent opportunity, a case study in practically applying almost every topic covered so far: data gathering through APIs, data engineering and hypothesis testing. A week or two ago, I started planning this post, with a clear four step plan: 1) define a question and metric to use, 2) gather the data, 3) perform the appropriate test and 4) try to quantify if it was a ‘big deal’.
Step 1: Define your question
Are the effects of climate change demonstrably real? Can I rigorously demonstrate that it is ongoing? How do you approach a topic like this with someone who’s skeptical, but not necessarily mathematically illiterate? The first thing I thought was that you couldn’t make an argument that actually referred to climate science or future predictions in any way: you would have to find a simple, concrete metric whose effects could already be seen. The climate is, after all, a complicated system and the people who can truly claim to have a comprehensive sense for how how it operates or how it might evolve are few in number. The rest of us would do well to listen to them, but we aren’t really in a position to evaluate their work. You can’t argue with something that’s already been observed, though.
Similarly, the metic should itself be simple and transparent. Weather systems are, again, complicated! Any argument that refers to things like particular storms or droughts is going to necessarily be opaque and uncertain. But something simple, like average temperature or total cooling degree days in a season, is undeniable. You may not know exactly why it’s the case, but you can tell if summers are hotter now than they used to be by looking at a thermometer, no modeling required. For practical reasons that will be clear when I discuss gathering the data, I opted to look at two simple temperature related metrics: average max temperature and number of days in a year over 90 degrees F.
Lastly, it occurred to me that there was a bit of tension between the size of the sample used (and therefore the confidence in the sample) and the magnitude of the effect you might be looking for. On the one hand, the size of your confidence interval in any random sample is going to depend on the size of the sample, and the climate is a relatively noisy data set: temperatures don’t just vary from month to month, but significantly from year to year, particularly with large climate phenomena like El Nino years. Without a large enough sample of years, you’d likely be unable to reject the hypothesis that warmer temperatures weren’t just the result of random fluctuations. On the other hand, the noticeable effects of climate change were likely to have been very small up until fairly recently: more than half of all carbon emissions have come in the last thirty years, and there is a lag between the emissions and warming.
Simply put, on the timeline of carbon emissions and global warming, the 1990's were a lifetime ago. Before looking at the numbers, I would have been surprised if there was no evidence of warming in the last few years, but I would have been equally surprised if warming had been noticeable already in the nineties. I settled on comparing the last decade with the last fifty years of the 20th century. Ten years seemed to me be long enough to capture enough variance, to avoid being influenced by one El Nino season and short enough that I wasn’t going to capture a number of years prior to substantial warming in my test. Would it be a large enough sample that I’d get a significant answer in the hypothesis test? We would see.
Step 2: Gather the data
I started this little project thinking that gathering decades worth of climate data across numerous cities would be easy. Imagine my surprise when it turned out to, well, not be super easy.
See, the best repository of climate data is, theoretically, entirely public domain. The National Weather Service oversees the operation of an enormous weather data collection network: on land, on buoys at sea, from weather balloons. Its operation is publicly funded and as such the data is publicly available. Numerous private weather services depend on this data: some of them collect their own data in addition, but the vast majority of measurement comes from the NWS. Anyone can access their historical climate data online through the National Oceanic and Atmospheric Administration’s (their parent department) portal, either by requesting and downloading individual data sets in pdf or csv form or through NOAA’s API.
All of this data being publicly available, I assumed that gathering it would be relatively easy, but it turned out to be a little more cumbersome than I had imagined, for a couple of reasons. For starters, the documentation on what data they have available and where it lives is not great. There isn’t a master document of datasets and datatypes available within them. Instead, to figure out what datatypes are available or what the codes for them are, you need to query the API, which turned out to be a little stingy. The API will only return 1,000 lines at a time, a number which may seem like a lot, but turned out to be really limiting. Their ‘Daily Summaries’ dataset turned out to have around 1,600 datatypes, so even something as simple as figuring out what was available and choosing what data to pull requires multiple calls to the API.
Actually finding particular pieces of data turned out to be a bit of challenge. One variable I considered using was the total number of cooling degree days in a season as that has an easily interpretable economic consequence in, well, the cost of cooling. The NOAA website teases the availability of this data in their Local Climatological Data section, which has detailed local measurements for many measuring stations across the country. You can manually click through to find the measuring stations on the website, but accessing this information in a scalable way requires finding a way to pull it through the API, since there are many such stations and there is so much data in each station report that you can only pull a limited amount of time at once — getting multiple years worth of data in one go is not possible. I eventually gave up trying to find where this data lived in the API. The ‘Daily Summaries’ dataset didn’t appear to have daily values for cooling degree days, only long term averages, and the station IDs I found in the Local Climatological Data section weren’t present in the dataset (indeed, the station IDs in the LCD section followed a different format). I’m sure there’s a way to get to the LCD data through the API, but I’m not sure what it is.
Another frustrating aspect of working with so many datatypes with so little documentation was that it was hard to tell if a certain datatype would be available for a given city at a given time. Some datatypes dropped in and out of the time series. I saw for instance, a series of datatypes related to temperature: daily minimums, daily maximum and daily average temperature. The average temperature seemed like a sensible metric to use, but, to my dismay I discovered that that the average temperature datasets only seemed to go back to the 90’s or so! I ended up using the maximum temperature dataset, which was more complete.
The other major roadblock was the limit in API calls. As I mentioned, each API call could only deliver 1,000 lines of data, even though there were more than 1,000 different datatypes, so it isn’t even possible to pull one days worth of complete data for one city in a single pull! Even if you wanted to focus on just a handful of datatypes, however, you would still end up running into the 1,000 line limit pretty quickly, because each city has multiple measuring stations and data like the temperature isn’t stored aggregated across stations, but on a station by station basis. Smaller cities may only have had a couple of stations, but the larger cities might have twenty or thirty.
So, pulling just the minimum, maximum and average temperature in one city for one day may require around 100 lines, meaning you could at most reliably pull 10 days at once for one city or maybe one day’s data across ten cities. Pulling those numbers for a few cities across decades would require many different API calls, an issue which can be handled with the judicious use of loops in Python, until you run up against the NOAA API’s daily limit of 10,000 calls. Again, this number sounds like a lot, but since you can only get a couple of days data from one city in any one API call, building out a dataset for numerous cities going back years quickly exhausts the limit. I ended up having family members sign up for more API keys for me. My original notion had been to try and pull relevant numbers from a large collection of cities, as many as I could really, and I still may, but the limit meant that in the last week I was only able to pull numbers for around a dozen (I started with the biggest cities in the country).
Step 3: hypothesis testing.
I ended up with a 70 year dataset for each city which I then compressed into year by year views in two different ways. In one I simply averaged the daily maximum and minimum temperatures to find yearly averages and in the other I counted the number of days that got hotter than certain threshold temperatures. Simply graphing these values is already visually suggestive. Take LA, for instance. Here are the number of days LA experienced each year where it topped 90 degrees along with a trailing ten year average:
It’s noisy, the number jumps around, but there’s a trend here, as you can see clearly in the trailing average line. Between 1950 and 2000, LA typically saw around 17 days a year that topped 90 degrees. In the last decade, however, that number was just shy of 27 days. A similar, though somewhat less extreme story is told in the average daily maximum, which ticked up about a degree in the last decade compared to the last half of the 20th century:
Suggestive as these graphs are on their own, they’re not quite sufficient. After all, even without any long term trends, the year to year variance is pretty high — relatively hot years are frequently immediately followed by relatively cool ones. If there was no long term trend at all, you might still see a couple of hot years in row due to chance. How can you be certain that you’ve observed the effects of change in climate rather than just the natural, random fluctuations of weather systems? To try and build some confidence here, we’ll try to quantify our certainty with a statistical hypothesis test.
The appropriate sort of test to use here is between two sample means — essentially I’m going to look at the sample from 1950–2000 and the sample from 2008–2018 and ask, what are the chances that these samples are from the same ‘population’, that is, that the random variables they represent have the same true mean and the difference I’m observing is due merely to random fluctuations. My test statistic is the difference of the two sample averages and the standard error for this statistic looks like this (no reason to assume that the two samples come from populations with the same standard deviation):
Intuitively, this formula should make sense. If my samples have high variances, then the difference between the sample averages seems like it would be more likely to be the result of chance. The test statistic’s standard error will be correspondingly higher and will present a higher threshold to rejecting the null hypothesis that the samples come from the same population. Counterbalancing this is that if my sample sizes are large, I’m more confident that my sample averages are close to the true population mean. In the context of climate data, this sort of error presents a pretty high bar to clear: I’m working with relatively noisy, high variance data, and my sample sizes are limited.
Even so, in the case of LA, these tests suggest that something is going on:
The average daily maximum temperature was a bit more than a degree higher in the last decade than it was in the last half of the 20th century, a difference you would only expect to see happen by chance about 1% of the time. There were on average 9 more days that topped 90 degrees in the last decade than in the last half of the 20th century, a difference you’d expect to see by chance less than one time in a thousand.
These sorts of numbers suggest that there has been some sort of long term change to the climate, that the population of prior years and the population of current years have different averages. Crucially, a test like this on its own can’t tell you why that is the case, it can’t verify any particular climate model or predict what temperatures will be like next year. A skeptic might suggest that there is something else going on, some non-human-caused factor that is causing weather patterns to shift. This test can’t rule that out, but it can rule out that there has been no change at all.
The other cities I’ve pulled climate data for (New York, Houston, Miami, Phoenix, Philadelphia, Chicago, Dallas and few others) have largely told the same story, sometimes more extreme versions. Here’s the number of days above 90 in Miami:
Despite my Northerner’s assumption that Florida is just one big swamp, Miami summers actually used to be pretty mild. In the sixties, Miami typically only saw a handful of days above 90 and in 1965 there were actually no days above 90. The warming trend was already noticeable in Miami by the 1990’s, but even including the nineties, the average number of days above 90 in Miami in the last half of the 20th century was only 24. In the last decade that number has shot up to 61, a trend so extreme and distinct as to happen by chance only one time in every ten million.
Not every city presented such an extreme example. Take a look at the average daily maximum temperature in Dallas, for instance:
The 2008–2018 average did actually tic up a bit more than half a degree relative to the 1950–2000 period, but in the case of Dallas, the magnitude is small enough relative to the year to year variance in temperature that you can’t rule out chance. Philadelphia was a similar story. Taken together, however, it looks like a pretty compelling story. Temperatures were up substantially in a number of cities. In a few cities temperatures had maybe risen, or maybe there wasn’t really a trend. In no cities did it look like temperatures had come down.
Step 4: quantify the human impact.
The last step is in some ways both the hardest and the most important. Even if you felt you had comfortably demonstrated an uptick in temperatures, you would need to answer the big question: is it a big deal? How much does an extra degree or an extra 10 or 20 days above 90 really matter? Even though there are a host of quantifiable impacts, this is actually sort of a hard question to answer. Higher temperatures impact us in many ways, from direct costs associated with air-conditioning to less obvious associations. Higher temperatures and heatwaves are closely associated with hospitalization rates for cardiovascular and respiratory complications in addition to heat related illness, particularly in at risk and elderly populations, so another 30 plus days above 90 a year in Miami can be credibly credited with numerous deaths. (In something of a double edged sword, it looks like the magnitude of the effect of heatwaves on hospitalization and mortality rates has come down in recent years, probably owing to higher coverage of air conditioning. So, while the danger of any particular heatwave has been mitigated, it is only because costs, energy use and related carbon emissions have gone up.)
Some impacts are hard to measure independent of all the other factors that influence them. Higher temperatures are associated with lower agricultural yields, which is consequential both to agriculture in economic terms and to consumers who face the consequences of less production. (I was living in London last year when high temperatures and draught hit European farmers hard, causing prices to rise dramatically, partly contributing to a shortage of French butter and, very frighteningly for me, leading to low barley harvests putting my beloved Scotch whisky at risk.) While conditions have deteriorated, however, agricultural techniques have improved, so outside of major heatwaves and droughts, it’s hard to tease out the effects of climate change on its own.
Demonstrating that there has been climate change therefore proves to be an easier task than trying to estimate the cost associated with that change. My plan is to continue to collect data and build up a detailed map of temperatures across regions and then map on some of the easier to measure costs. That will have to wait until the next post though.
