How accurate are “day ahead” energy demand forecasts?
The EIA recently published a new dataset on hourly electric demand across the US grid. As part of this, they published the “Day Ahead Demand” forecasts put out by Balancing Authorities to predict the next day’s energy demand in their areas. I thought it would be interesting to analyze the accuracy of these forecasts, to see which areas of the country are best at forecasting their energy deman!
But first: what is a balancing authority?
Balancing authorities are organizations tasked with “ensuring supply and demand are balanced in real time” in the US energy market. Balancing authorities can be either utilities or RTOs, regional transmission organizations. To balance supply and demand, they issue orders to generators to change their output as needed. There are 66 balancing authorities certified in the United States.
How do balancing authorities forecast demand?
Balancing authorities use algorithms based on historical demand and key variable factors such as temperature, wind speed, and precipitation to predict day ahead demand. Sometimes, the forecasts will even incorporate the impact of major events — for example, if everyone in the area is staying up late to watch election results.
And, forecasting demand has gotten harder with the growth of distributed energy resources, such as solar panels. Technically, the grid does not actually care about day ahead total energy demand. A grid cares about day ahead energy demand from the grid. These values used to be mostly synonymous, but now can be very different, as consumers of energy also tap their home solar panels. And, given that the energy output of solar panels is highly dependent on weather, this just makes grid demand even more variable based on weather!
What are day ahead forecasts used for?
Well, these forecasts are used to plan production and inform purchases in the “Day Ahead” energy markets. These markets are run by the RTOs, and allow retailers of energy to procure supply from generators. Day ahead markets are used to set the production schedules.
What happens when these forecasts are wrong? Well, then you go to the “Real Time” energy markets, which are used to move production around. In general, though, missing on demand forecasts will cause higher prices and lower grid reliability. Day ahead markets are typically cheaper and less volatile than real time energy markets, and obviously under-allocating capacity to certain areas can cause brownouts / blackouts if the grid is not able to adjust real time.
So, what does demand accuracy look like across the country?
To see this, I analyzed the “hourly accuracy” of each Balancing Authority in the six-month period from January 1, 2020 to June 30, 2020.
However, most of us don’t really know how to understand what “good” looks like in this market. Is an average 2% miss good? 5%? So, to provide a benchmark, I calculated a “Naïve” forecast for each day as well. The “Naïve” forecast was calculated by just taking the previous day’s demand at the same hour. In my mind, this was the simplest algorithm you could have to forecast demand. So, below are the results for the top 30 balancing authorities in the US (sorted by total demand, in MW, over the period of Jan 1 2020 to June 30 2020).
As you can see, most of the balancing authorities did indeed outperform the “Naïve” forecast, some significantly (ERCO, which is Texas, TVA, which is Tennessee, and ISNE, which is the Northeast, were three of the larger standouts). However, there were two that severly underperformed — PSEI and FPC. I am not sure what is going on here, but I will note that both of these had interesting forecasting behavior, in that they underestimated demand 100% of the time over 6 months. For every hour, of every day, for 6 months, they forecasted too low. This is obviously intentional. So, I think this may be a strategic decision, and not necessarily a forecasting miss.
This view is by balancing authority, but what does this picture look like across the country? Well, as you can see, I tagged each Balancing Authority above by their “major state” — this means they were the top BA in the state (those that don’t have states were second or third in total demand in the states they served). Here is what accuracy looks like across the country.
Why is this so important?
Well, as the renewables portion of our energy supply increases, the energy supply becomes more variable (due to the variability of wind and solar). And, as we discussed, the demand side is becoming more variable as well, due to the use of distributed energy resources. So, matching supply and demand is a much different problem today than it was 10 years ago.
Luckily, there is real innovation happening in this sector. A few areas are particularly exciting, in my opinion: 1) More robust data 2) Real-time trading and 3) Demand response.
More robust data includes both better external data (ultra-local weather feeds, etc.) and better in system data (through smart meters that can provide a more nuanced view of consumer use and consumer generation to utilities). These more robust data sources can give more raw material for improved forecasting techniques, relying on AI.
Then, more finely tuned trading systems can allow system operators to procure short-term reserves and contingency when forecasting is (inevitably) wrong.
Finally, demand response systems allow utilities to work with their users to adjust demand to meet supply, instead of the other way around. These systems exist at scale today, but will need to continue to become more granular and faster.
All these innovations, and others, will be essential in building a resilient grid in the future.
