A Curious Case of Temperature Fluctuations

January 2017 has been an interesting month. Our country is more divided than ever on issues ranging from women’s rights to trade to how humans have contributed to climate change.

When I returned to NYC from Seattle after Christmas break — prepared for the full-fledged East Coast winter — I was taken aback by the weather. The temperature appeared to be here, there, and everywhere; one day it is below freezing, and a few days later the sun is shining and we are well into the 60's. Lets take the week of 1/8–1/14. On that Monday the 9th, the lowest recorded temperature was 14 °F. Later that week, on Thursday the 12th, the temperature reached a whopping 66 °F. That’s a differential of 52 °. While I am not a meteorologist nor did I study climatology, this seems like a large change for mere 72 hours.

I wanted to investigate this further. Just how typical are such large temperature differentials? And are we observing larger differentials now than we have in the past?

I requested temperature data from the U.S. Climatology Network for a Station ID 305801, located at Central Park, right in the middle of Manhattan. The data spans from 1/1/1950 to 12/31/2014, and includes daily maximum and minimum temperature measurements, measured to the nearest degree (°F). You can find this data here.

To get a feel for the data, I calculate the intraday temperature differential, and then calculate the average intraday differential by month for 1950–2014. This data will tell us how much, on average, the temperature fluctuates within a day in a given month.

Data ranging from 1950–2014

Lets plot this and see what the data looks like visually:

The average temperature differential is highest in April and May and is lowest in December and January. That’s interesting. The temperature fluctuates the most in transitioning months between Spring & Summer.

I assembled similar data, selecting the period of 2004–2014 (10 year span) and 2009–2014 (5 year span). Here is the data in tabular form.

Data for 2004–2014

Data for 2009–2014

Now lets plot these three averages: 1950–2014, 2004–2014, 2009–2014.

There are a few observations to be drawn from this chart:

1. The average differential for 2004–2014 and 2009–2014 is actually less than the average differential over the whole time span in question.
2. The data follows the same trend: the differentials are the smallest in January and December, and increase in April-June.

We are getting somewhere now — the average differentials do not appear to be increasing.

Let us form a coherent hypothesis and flex the statistical muscle to test it further. The question is this:

If we slice the data by week (i.e. weeks 1-53), are the average temperature differentials observed within a given day in that week in a time period from 2004–2014 (10 years) statistically different from the 1950–1980 (30 year) period?

In statistical terms, the hypothesis is such:

I will be using a two-sided t test with a significance level of 0.10. To do this, a few quantities need to be calculated:

1. Standard Error
2. Degrees of Freedom
3. T-statistic

I don’t want to dive into the math too much (given my limited knowledge of statistics and probably your limited interest in the subject), but in a nutshell what we will do is determine the T-statistic, and together with the Degrees of Freedom use the T-statistic to calculate the P value from the two-tailed T-distribution. If the calculated P-value is less than the significance level (0.10), the difference of two means is statistically significant.

Okay, so what do the results look like? Check out the AirTable below.

The weekly mean (u1: 1950–1980) is statistically different to 0.10 level from the weekly mean (u2: 2004–2014) in 24 of the 53 weeks. These are the 24 weeks in question, represented visually here:

What are some observations to be drawn from this graph? Most of the weeks that display statistically significant difference in differentials are clustered in select time periods: end of May — beginning of July (roughly weeks 21–28) and October (roughly weeks 40–45).

It’s also interesting to note that the mean fluctuations, when statistically different, are always lower for 2004–2014 span than they are for 1950–1980.

Last but not least: instead of looking at intraday fluctuations, what if we look at the maximum intraweek fluctuations? That is, the question is:

If we slice the data by week (i.e. weeks 1–53), are the maximum temperature differentials observed within that week in a time period (averaged) from 2004–2014 (10 years) statistically different from the (averaged) 1950–1980 (30 year) period?

This is getting a little verbose, so let me quickly explain what I mean. Suppose we have two weeks in question:

The max differential in week 1 in 1950 is 66–31=35 °F; for week 2 in 1950 it is 61–16=45 °F. I find the max differential like this for every week from 1950–1980, and then take the average.

Running a similar test as above, the results are as follows:

The two averages are statistically different for 20 weeks, represented visually below:

Again it is clear that when the two averages are different, the 1950–1980 data point is higher than the 2004–2014 one. What’s curious is that we again observe the clustering phenomenon, as during the previous discussion.

These are interesting observations for many reasons, but for me this is a lesson not to jump to conclusions. It is easy to form a judgement based on singular observations or personal sentiment — and generalize from there — but these generalizations are often simply incorrect.

In conclusion:

a. We are not observing larger differentials now than we have in the past

b. Historical differential averages are actually higher than the average for the last 10 years