How new AI capabilities are transforming our grid systems and making it more “flexible”

In April 2024, the UK Government announced backing eight AI projects to speed up its net-zero ambitions. This Government initiative reveals how the digitalisation of the grid is becoming a priority for public authorities. A system that remained for a long time fairly analogue is now undergoing a massive transformation. To understand why this critical digital transformation is happening now, we first need to turn to the challenges faced by deploying renewable energy on our current grid system.

Following global geopolitical tensions with Russia and record highs in energy prices, Western nations have announced ambitious plans to accelerate the deployment of renewables. Large-scale investments in renewable were initiated following the 2015 Paris Agreement, but recent events have considerably increased the pace of deployment.

To give some perspective, globally over 1,000GW of wind and solar is in operation today. It took us around 40 years to reach this milestone. With current plans, it should take us under five years to reach the 2,000GW mark.

Unfortunately, it seems we have reached a bump in the road.

According to experts, we are at a point where our systems are not capable of connecting new renewable projects at the same pace.

Indeed, the installed capacity of renewable energy outgrew the capacity of the distribution grid, impeding operator’s efforts to effectively transmit energy to the final consumer. The cause is that renewables are not suited for the current grid system due to their intermittent nature and different base loads.

These characteristics are problematic because consumption demand is not linear. Our consumption resembles a “duck shape” curve, where we experience huge spikes in demand at a time when renewables are not necessarily capable of providing the supply. These spikes overload our current distribution infrastructure and lead to a situation where most of the renewable energy is lost in an attempt to reduce volatility (i.e curtailment).

In addition to the challenge of dealing with fluctuating supply from renewable, power systems now need to also support multi-directional flows of electricity between distributed generators, the grid and users. The rising number of grid-connected devices, from electric vehicle (EV) charging stations, to residential solar installations, makes energy flows less predictable. Meanwhile, links are deepening between the power system and the transportation,

industry, building and industrial sectors.

The result is a system where we need to have better prediction of energy supply and at the same time the capability of orchestration between all the different elements at play in a decentralised grid system.

Building that grid system of the 21st century, we need to be able to optimise the management of energy on our current grid system before considering any expansion.

Luckily, these challenges arise at a time when the capabilities of AI applications are progressing exponentially. While ChatGPT had its momentous release in 2023, GPUs and other semiconductor capacities have also improved considerably. AI tools are now serious contenders to solve highly complex problems such as the stabilisation and the optimisation of the grid system.

AI is poised to play a pivotal role in the transformation of our grid. New data from IBM, unveiled at DISTRIBUTECH International, indicate that almost three-quarters of energy and utility companies surveyed have implemented or are exploring using AI in their operations. Consulting firm Indigo Advisory has counted more than 50 possible uses for AI in the energy sector. The company estimates that 100 vendors have already introduced AI solutions into their products and Indigo estimate that the market for AI is now worth up to $13 billion in the energy sector alone.

Many fascinating AI applications are being considered in this space, but currently, the most fundamental use case is to bring more “flexibility” to the system.

The energy flexibility market refers to the ability of energy systems to adjust their generation, consumption, or storage patterns in response to changing conditions such as demand fluctuations, supply intermittency, or price signals. It is in this complex new decentralised

system that AI will play a critical role.

Below we will examine different forms of AI applications used to improve our grid system and provide more flexibility. We will see how high computing capabilities have now become essential to anticipate price volatility, fulfil high-frequency orders, optimise energy flow and improve stability in a system that has become ever more complex.

Weather forecasting

The first use case that AI is starting to have a material impact on is forecasting weather and the generation of renewable energy.

As mentioned above, because sources like wind and solar are so volatile, AI algorithms are now essential to analyse weather forecasts, historical generation data, and real-time conditions. These applications enable energy providers to predict how much renewable energy will be available, allowing for better balancing of supply and demand.

These prediction tools are also critical for many actors involved in energy markets. Energy traders involved in “day ahead” markets can now better anticipate prices and thus be able to place optimal bids.

In day-ahead markets price anticipation is essential as the market is made to balance electricity supply and demand before production and delivery. Capacity to anticipate prices provide an opportunity for market participants to plan their electricity generation and consumption in advance. It helps to optimise resource allocation and minimise costs by matching supply with forecasted demand. It allows utilities and consumers to hedge against price volatility by securing electricity at known prices.

To address this, Google and its AI subsidiary DeepMind developed a neural network in 2019 to increase the accuracy of forecasts for its 700 MW renewable fleet. Based on historical data, the network developed a model to predict future output up to 36 hours in advance with much greater accuracy than was previously possible.

This greater visibility allows Google to sell its power in advance, rather than in real time. The company has stated that this, along with other AI-facilitated efficiencies, has increased the financial value of its wind power by 20%.

Swiss manufacturer ABB has developed an AI-enabled energy demand forecasting application that allows commercial building managers to avoid peak charges and benefit from time-of-use tariffs.

Algorithmic trading

AI is not only used to forecast accurate prices, but to improve day-ahead trades. Algorithmic trading has become an essential tool for “intraday” settlements.

With the increasing amount of renewable intermittent production, it has become ever more challenging for market participants to be in balance after the closing of the day-ahead market. The intraday market offers opportunities for market participants to optimize their positions and mitigate risks associated with imbalances.

But in an environment with huge variability and ever-growing data points to process, operations that were historically done manually now require AI tools to optimise trades. This method of trading has become increasingly popular in recent years due to its ability to quickly analyse large amounts of data, and execute trades at high speeds, all the while improving reliability by removing human error.

A variety of tools are already on the market, including GE’s Alpha Trader, which uses AI to predict energy generation and pricing and pair it with a portfolio’s specific risk profile.

Meanwhile, Fluence’s bidding software Mosaic uses the technology to develop optimized bids based on forecasted prices. Tesla’s Autobidder focuses specifically on battery assets, including price and load forecasting.

Companies like Enspired and Esforin are optimizing renewable energy assets and storage by maximizing revenues of flexibility by automating trading on wholesale markets.

A British firm, Arenko, has created what amounts to an AI-powered autonomous operating system for grid-scale battery installations that allows operators to manage large electrical inventory super efficiently. Their solution is now used by more than 25% of batteries in the UK.

Gaiascope, for its part, said it forecasts more than 3.8 million data points on the grid daily, and that its bid optimization and forecasting software has the potential to increase returns for clean energy generators by over 100%.

Demand side response (DSR)

As mentioned above, our grid system is confronted with several challenges. Electricity demand increasing whilst traditional generation is in decline. Variable output and demand throughout the day cause stress on the system. To solve this problem, AI solutions are put to work to participate in what is known as demand-side response (DSR) programs.

When stress on the grid is high, instead of increasing electricity supply from the grid to meet consumer demand, grid operators instead turn to energy users to reduce their demand for electricity, ensuring supply-demand balance. To incentivize energy users to provide this needed energy flexibility, grid operators offer lucrative payments (in the form of demand response payments) or bill savings for organizations that can reduce or shift their energy use away from hours of highest stress on the local electricity grid.

With new AI capability available, there is a fundamental change occurring in how the new distributed grids will operate. In the near future, DSR players will transform the way energy is consumed. Rather than a unilateral (top-down) flow of energy, the new grid is incorporating new actors with demand response functions who are incentivised to adapt to real-time price signals. In the long run, the concept of bilateral energy flows in which DERs (distributed energy resources) both consume and store energy will become common practice and participate in the flexibility market.

The technology layers for this new grid system are rapidly evolving and open up the market to new opportunities.

At the grid scale level, battery companies such as Arenko, TerraLayr, Terra One and Field are some of the largest actors in this DSR market. They use tech solutions developed by Flower, Enspired, Esforin or Elum energy or NextKraftwerke, to optimise the management of assets.

At a smaller scale, different actors are bringing flexibility to residential and commercial building owners.

Full-stack installers like 1KOMMA5 and Enpal on the B2C side are entering the customer with solar systems installation and aim therefore to monetize the flexibility of their asset. Solutions like Elyos or Pearlstone Energy incentivise I&C (Industrial & Commercial) consumers of energy to turn down or shift non-essential electrical plant or devices in their buildings, during times of stress on the grid. Thus allowing I&C to become price-efficient by reducing demand during periods of high prices in the market or by shifting energy-intensive operations to a time when prices are lower. Their solution also allows businesses to get a share of revenues secured through participation in various NGESO & DSO flexibility services.

Also in the C&I space, companies like InRange maximise the use of rooftop solar for tenants’ self-consumption and sell the excess electricity through PPA (Power Purchase Agreements) to off-takers looking to secure stable prices.

The demand-side response could provide an important contribution to managing the security of supply and cutting energy consumption. It offers a cheaper and greener alternative to building new generating capacity and could make a meaningful contribution towards the security of supply.

In effect, the DSR Aggregator will enable property owners and asset managers to access their assets’ untapped revenue sources while meeting energy-saving demands, cutting costs, and contributing to the reduction of carbon emissions.

New energy provider

AI is also used by new forms of energy providers deploying solutions to allow customers to access spot energy prices and mitigate price volatility. Consumers equipped with smart meters can now access new forms of dynamic pricing.

These energy companies optimise electricity use with the help of smart technology. Their tools analyse consumer behaviour and optimize their consumption using smart algorithms.

Digital energy retailers like Greenely, Ostrom, Tibber, Motkraft and Rabot Charge have a B2C offering dynamic energy tariffs and build software stack to manage the loads and maximize flexibility revenues.

Other companies like UrbanChain, Reel Energy and Trawa also use AI to optimise energy price offered to their customer. They act as a platform between developers striving for stable revenue streams to improve the bankability of their projects and consumers looking for stable electricity prices.

In a decentralized energy distribution network, individual homes will also become actors in the supply and will be able to participate in demand-side response with Virtual Power Plants (VPP). Companies like David Energy, Fusebox or Fever are examples of these new software programs bringing flexibility to local homes.

New data management tools are now entering the market to allow energy to be generated and managed locally. Companies like Arcadia, Electryone, or Solarize provide software that allows shared building supply, dynamic tariffs, and automates on-site PPAs.

Market opportunity

On 8 July 2020, as part of the European Green Deal, the Commission presented an EU Strategy for Energy System Integration. The strategy committed to adopt a Digitalisation of Energy Action Plan to develop a competitive market for digital energy services that ensures data privacy and sovereignty and supports investments in digital energy infrastructure. The EU intends to mobilise up to 300 billion euros of investments by 2030 into DER.

This EU initiative is setting out a strong dynamic and putting in place a framework for the implementation of new local legislation and other public support to accelerate the digital transformation of the energy system and promote flexibility.

Five focus areas are suggested, of which one is to develop a European data-sharing infrastructure to create a competitive market for energy services that value demand-side flexibility and support planning and monitoring of energy infrastructure. Another is to empower citizens by providing them with tools for participation in the energy markets, tailored data-driven services and implementing reskilling and upskilling pathways.

This action plan focusing on the flexibility market is one of the many legal initiatives pushing the digitalisation of the grid infrastructure across Europe. EU member states are not all advancing at the same pace. But the ambition is to unify the European energy market with the same standards and tools. The ultimate goal being to improve energy efficiency by 11.7% by 2030

The Future Energy Scenario (FES) modelling published in July 2023 suggests that the volume of “pure” DSR could double within two to three years as the value of flexibility sharply increases. The UK Government projects that low-carbon flexibility, including demand side response, could reduce the cost of running the electricity system by £10 billion per year in 2050 (around 14% of the system cost), with a cumulative cost saving of £30–70 billion between 2020 and 2050.

Advancements in technology in the utility vertical and the introduction of regulatory frameworks are expected to increase AI tools and other smart grid deployment, which is estimated to drive market growth.

As an investor, this market is particularly exciting as we are at the beginning of this digital transformation. These profound transformations will not only have an impact on everyday people with the anticipated decrease in electricity prices, it will make our overall system more resilient, and most importantly it will accelerate the much-needed green transition.