Barrow, Alaska — Why Outliers & Anecdotes Create Poor Science
And why it is time to move on already…
On the Northern edge of the continent, sits the United States’ most Northern city — Barrow, Alaska. Well at least it did.
No, it wasn’t destroyed by global warming, the ever-increasing population of angry polar bears, or even a bad filtering mechanisms at the NOAA — though all of those are a potential issue. Barrow was renamed to Utqiagvik a little over a year ago. In was a provably man-made event by all accounts. And so Barrow is no more…
Not long after, Utqiagvik began disappearing from the NOAA monthly climate report. We are assured this had nothing to do with the name change and everything to do with global warming. Neither of these two causes has anywhere near sufficient data to be proven or dismissed. What is understood is that average daily temperature in this region had increased on a year-over-year basis and was now being filtered from the report.
Utqiagvik was being treated as an outlier. Or stated bluntly, data that scientists (or those who claim to be) ignore because it looks weird. If that sounds oddly unscientific, well… you are getting the point. The algorithms (or those things people claim are algorithms but are really rather arbitrary rules) determined that Barrow’s data was unreliable due to the size of the change and so, purged it from the data set.
It is all explained here (warning: not a page turner):
Or you can just read my quick summary — things change. People move equipment. No one is really trying that hard at the science-thing and government sucks at keeping the equipment working, especially when you are talking about instruments spread the world over. The easiest way to fix that is to make a rule that if something changes too much — throw it out. If that sounds oddly unscientific, well… it gets better… worse? … more complicated!
- How many of these outliers are there? Good question — I haven’t found an audit list anywhere online.
- Who determines how much change is a problem? Good question — the blog indicates numerous rules with clear geographic bias…
- Is this good science? … No, but don’t worry, it gets worse…
One of the least scientific measurements possible is an average. Weighted averages are the absolute worst, but averages with outliers removed run in close pursuit and those with subjective filtering close the gap almost entirely. And no one has declared that the NOAA’s global temperature isn’t weighted. In other words, this is just bad, bad science.
But wait, surely there are practical considerations that force scientists to remove outliers to better enable them to understand trends in nature?
Perhaps. BUT — best practice would be to identify outliers using a secondary measure, NOT the one you are actually averaging. In other words, filter the results based on a recorded move or outage, anything but a temperature change would be an improvement. Secondly, you want to use consistent rules and certainly not bias them using a variable linked to what you are measuring. Lastly, do NOT change them over time (not if you value time series, more so if that is what you are benchmarking against).
So how does this get worse? Turning a selected outlier into an anecdote. The Washington Post is really good at this — although in this case, the scientists started it.
Anecdotes can be great ways to connect to the story of the data, BUT you want to use anecdotes based on typical data NOT outliers. An outlier is supposed to be defined as an exception, a mistake, an error. Anecdotes, used well, are interesting but also transferable. Those based on outliers can almost never be the latter.
So is all this to say that Barrow is not warmer right now than past years? No. It very well may be. Is this to deny global warming? No. But nothing we talked about proves it either. This is the story of an “outlier”, defined with poor subjective criteria, and turned into an interesting but unscientific “anecdote” — all in the name of tracking the weighted average of another average randomly sampled across time and space with numerous techniques and technologies.
You can draw what conclusions you want. I will wait for satellite coverage to finally create some legitimate and consistent measurement and a couple of decades to build up some real time series.
Yes that might take a while, but progress counting polar bears is already being made. With any luck, real science will soon confirm Polar Bear populations on a regular basis. Then we will know whether they are shrinking or growing, the debate can evolve into just how angry they are…
Until then — avoid big averages, include outliers whenever possible, and keep your anecdotes to the typical use cases, not the ones you otherwise reject as noise. If nothing else — stop calling it Barrow.
