Energy Has a Data Problem: Three Absurdities Preventing the Transition to a Decarbonized, Smart Grid

Nancy Hersh
The Arcadia Source
Published in
6 min readNov 11, 2021

Ten years ago, most people had no idea where their energy came from, and few cared. Now, solar panels are an increasingly common sight, in-home batteries are on the rise, and companies are publicly reporting their carbon footprint.

All of this is undoubtedly exciting. As the Chief Data Officer of a climatetech company, I am thrilled that clean energy is finally at the forefront of our cultural and political zeitgeist. However, as a data nerd who has had the opportunity to analyze data from 125 utilities nationwide and 80% of US electric utility accounts, one of the biggest problems in our clean energy transition is incredibly underreported — how can connected devices operate at the cleanest and lowest cost times to optimize energy use? In other words, how can we make the internet of energy a reality?

The internet of energy would be able to turn energy from any device into information. It would provide real-time visibility and control into energy usage and sources on micro and macro scales. That would truly be a game-changer.

However, these new and innovative clean technologies that are quickly coming online are entering a system operating under a structure designed well over a century ago that will slow adoption, keep us from realizing their true potential, and (most importantly) prevent us from moving at the pace we need to combat climate change.

It’s clear to me that three absurdities within the legacy energy system will be inevitable roadblocks to unlocking the potential of the imminent cleantech revolution and electrifying and decarbonizing the economy once and for all.

Absurdity #1: People often pay more for energy than they have to because of inertia, confusion, and lack of awareness

First, the vast majority of US customers are in a state of inertia with their utility rate and accept the default rate that their utility set for them years ago, which in many cases isn’t the best rate for them.

They are reluctant to change for two reasons. They are either not aware that they’re eligible for programs that can lower or decarbonize their energy bill, or they’re worried that it might make their bill more expensive if they make a change.

Second, finding the right electricity rate can be incredibly complicated and confusing. The average US residential customer has multiple time-of-use (TOU) tariffs to choose from, designed to incent them to use energy at low-cost times.

Still, only a small percentage of customers are enrolled on these rates because the process is so difficult to understand. For example, roughly 70% of energy customers are eligible for non-standard rates, but less than 5% of customers switch from the default tariff assigned by the utility.

Third, customers can now leverage their energy bill to source 100% renewable energy and have it 5–10% cheaper than standard rates with the advent of community solar programs spreading throughout the country. Nevertheless, when customers think of solar, they still imagine rooftop panels, but there’s one problem: two-thirds of Americans can’t put solar panels on their roofs.

These households are often poor candidates for rooftop solar for one of three reasons:

  • They rent or live in an apartment.
  • They’re unable to finance their installation.
  • Their roof cannot support solar.

These households tend to skew towards people in lower-income brackets, and we must provide viable, clean options for these populations. However, our proprietary survey data shows that lower-income households are generally unaware of their best option — community solar.

Wading through the energy jargon to make sense of your options is challenging. It’s much easier for customers to tune out and accept the default settings from the utility — so they do.

Absurdity #2: Utilities often cannot access their own data (!)

Utilities are technologically behind the times because of their monopolistic business model — like most monopolies — they don’t have an incentive to innovate and maintain a customer-centric view.

For example, they still refer to their customers as ‘ratepayers,’ reflecting a view that customers pay whatever rate is needed to support the asset base of the utility. Utilities also often rely on third parties (like Arcadia) to unearth their own data because they can’t access it themselves! They’re reluctant to make improvements to their impenetrable old, brittle systems that would streamline data management and analysis because of the risk and time involved.

The inability of utilities to effectively utilize their own data is also why energy bills are incredibly ugly and hard to understand.

Not having the ability to access and analyze data will simply not be acceptable for the energy grid of the future. As we transition from a fossil fuel, centrally controlled energy system to a clean energy, distributed system, energy will be generated and deployed from various sources from rooftop solar installations, wind farms, backup battery installations, and nuclear power plants.

For new cleantech devices such as EVs and home storage to realize their promise, they must be able to “talk” to the grid to ensure that energy is deployed as cleanly and intelligently as possible. However, it is difficult for device manufacturers to work with the sprawling number of utilities throughout the country if utilities can barely “talk” to themselves.

In order to orchestrate the symphony of a multimodal energy system, our entire energy system will need to have quality data that illuminates:

  1. Supply: where energy is being generated, and it’s carbon footprint at any moment
  2. Demand: energy needs of every node in the system, with a granular time frame
  3. Pricing: transparent, simple pricing that reflects actual supply-demand dynamics

Absurdity #3: Customers are expected to do too much to make our electric grid more efficient

Energy is boring, at least to most people. And, people are busy! They should not be expected to plan their day around the economics of the grid. The clearest evidence of this is the very low rates of uptake for demand response programs.

Are we really going to ask someone to regularly consider when to do their laundry based on when they ‘should’ vs. when they ‘can’? Or are we going to expect the wave of new EV drivers to figure out when energy is cheapest and greenest to charge their cars?

No. The answer is no.

That expectation would add complexity and difficulty into people’s lives that are clearly not manageable.

We have the ability to develop solutions that do the heavy lifting for consumers automatically and facilitate the decisions they want to make. We can develop a smart energy system by leveraging machine learning and the increasing number of internet-connected devices.

We have the technology!

This system could allow customer control when they want it, but begin with intelligent defaults that optimize against cost-savings and climate-friendly preferences, whether that be saving money, using greener energy, or both. This could help consumers operate their devices at the lowest cost and greenest times or identify which tariffs are best for each person.

Looking Away Won’t Help

These absurdities with utilities might not be sexy enough to grab headlines, but these problems are ragged enough to make rooftop solar or a shiny new electric vehicle less appealing in practice.

Ensuring our new tech meshes with our legacy tech is a common problem in technological endeavors. It’s just that this time we aren’t trying to ensure that our operating software can operate on a dated monitor. We’re trying to transform how the largest piece of tech man has ever created (the grid) works in the incredibly short amount of time we have to make a meaningful impact on the climate crisis. No pressure!

Just because these absurdities aren’t alluring doesn’t mean they don’t exist, and by not looking at them, they won’t go away. Climatetech innovators need to be thinking about these hurdles to our grid now before they slow down the transition to a clean energy future and steal the precious time we have to combat the climate crisis.

--

--

Nancy Hersh
The Arcadia Source

Nancy Hersh is the climate-conscious, number-loving CDO at Arcadia.