Will AI revolutionize the distribution, production, and consumption of electricity to reduce carbon emissions?
By Paul-Adrien NICOLE, Data Scientist at LittleBigCode
Mastering the consumption and production of electricity is an essential task that scientists ought to solve in order to fight climate change. One of the most promising assets to tackle this issue is the development of smart electricity grids — also known as smartgrids — which can efficiently manage the electricity distribution in the grid. Using solar panels and domestic batteries is an opportunity for smartgrids because it can improve the electricity management by consuming more electricity when it is cleaner from an ecological standpoint. Besides, Electric Vehicles (EV) will foster its development as it can substitute domestic batteries.
To that end, we developed LittleBigCity, a smartgird simulator which can use Reinforcement Learning (RL) agents to efficiently act as consumers or producers. Our aim is to highlight that the use of AI supervised smart energy assets (as we will explain later) can reduce the grid’s dependence on weather conditions and reduce electricity consumption peaks, hence lowering global carbon emissions.
Sources of inspiration
Saint John Energy
As stated in [1] “the current electricity grids will experience a profound change in the coming years”, the term smartgrid embodies this innovation highlighting new ways to produce, store, and distribute electricity. To hasten this development, the International Smart Grid Action Network (ISGAN) holds an annual contest to promote smartgrid projects across the world. And the first prize of the contest in 2021 was the Smart Energy Project from Saint John Energy (a Canadian company).
According to the company, consumers can be equipped with smart energy assets such as domestic batteries, solar panels, EVs (which have a battery), and heaters. AI can optimize their behaviour so that electricity is produced and stored when the weather conditions are mild, as sun foster electricity production in solar panels and wind improve wind turbine yield.
Domestic batteries can also be charged when production is at its maximum and discharged during tough times. Such an improvement requires consumers to buy new domestic equipment, but it will undoubtedly reduce their total electricity consumption and the induced carbon emissions.
CityLearn technology
CityLearn [2] is the first open source smartgrid simulator using AI agents acting as an electricity consumer such as a factory or a private individual. It was created in 2020 by José R. Vazquez-Canteli and Dr. Zoltan Nagy, two researchers in the Department of Civil, Architectural and Environmental Engineering at The University of Texas at Austin. Their goal was to create a physically realistic model for smartgrid simulation.
They organized the CityLearn challenge to reward the best AI in their simulator according to several metrics (minimizing electricity peaks and carbon emissions). In their model, consumers are equipped with smart energy assets: solar panel, domestic batteries, heat pumps, domestic hot water and electric heaters. The agent can manage the power given to each of these equipments. A consumer producing more energy than he needs can send it to the smartgrid leading to potential coordination between the buildings.
LittleBigCity
While CityLearn focuses solely on the customer side, we took this limitation into account to introduce LittleBigCity. This technolgy is a new open-source simulator that creates a two sided smartgrid: the consumer side (inspired by CityLearn), and the producer side (similar to Johan’s recent article: Optimiser la production d’un ensemble de systèmes de production d’énergie avec du deep learning ?). We also added a way to see how the smartgrid changes in real time with Streamlit.
The consumer approach
On the consumer side, we tried to simplify CityLearn’s model as we only used 4 smart energy assets:
- Solar panels produce electricity if there is enough solar incident energy;
- Domestic batteries store and release energy according to the agent’s action;
- Electric heaters convert electric energy to thermal energy for heating;
- Heat pump cool or heat the building using electricity.
The consumer agent has to maintain its temperature within a certain range with its equipment while minimizing its electricity consumption. The consumption of these assets can be set by the agent each hour, and choosing which energy asset to prioritize under climate variations can be done efficiently using artificial intelligence (AI). This is a multi-agent system where one agent corresponds to one consumer.
The producer approach
On the producer side, the objective is to find the best energy mix between the four potential production modes (wind, solar, nuclear and fossil) to maximize revenue under several constraints:
- Wind and solar energy’s efficiency depends on the weather,
- Nuclear energy requires time (several hours) to start;
- All modes of production are capped by a specific percentage of the maximum building consumption;
- All modes of production have a different unit cost of production (per kWh).
The production side of the smartgrid is a single agent system.
How to define the reward function?
To efficiently train our AI agents, a reward function need to be defined for each side of the smartgrid. Agents will try to maximize the total reward throughout the simulation.
- The reward function for consumer agents should be infinitely negative if the building’s temperature goes outside of a certain range (usually between 19°C and 21°C) and otherwise proportionate to the opposite of the consumer expense:
Consumer reward function (D is the electricity demand from the consumer and p is its price
- The producer reward function should be infinitely negative if production is not matching consumption (within a small epsilon margin) and otherwise proportionate to the producer revenue:
Producer reward function (P is the electricity production).
Electricity market
Enriching our simulator with a producer enables the creation of an electricity market setting up a unit price for electricity. In today’s electricity grid, the producer sells electricity to the grid at his preferred price, whereas the consumer can only buy electricity (represented as the blue arrow in the schema). In a smartgrid, the consumer also has the possibility to trade its electricity (yellow arrow) thus influencing energy prices.
The economic aspect of this model is an advantage because it will allow the implementation of several economic features. It simplifies the creation of incentives in the smartgrid such as a carbon tax to shift the energy mix toward renewable energy.
The idea is that increasing the cost of production of non renewable energy proportionally to their carbon emissions will make the producer agent prioritize the cheapest and cleanest energy in order to maximize his revenue. This aspect has already been tackled [3] but too briefly as the paper is short and the code is not provided.
Another component of the simulator is its dashboard. It displays the current smartgrid state in the simulation to visualize the agents’ behaviour. The “State mode” describes the producer energy mix and different numerical information about each consumer (the consumption of its assets, its temperature and its revenue). The “Graph mode” lays out several charts datailing several aspects of the simulation: the energy mix, the revenue on the producer and on the consumer side, and consumer state. The dashboard can be seen here.
Like CityLearn, the smartgrid involves a Gym Environment simplifying the implementation of RL agents. One of our objectives is to observe autocurricula (the emergence of new behaviour) between agents as in [4]. Besides, we are still thinking about creating an open AI challenge to draw the attention of the community.
In conclusion, we anticipate that LittleBigCity will be just as beneficial to smartgrid researchers as CityLearn was. This new initiative may appeal to AI enthusiasts and those interested in resolving the energy and ensuing climate crisis, as the visualisation tool makes the simulator more accessible to them. The simulator can be found on our Gitlab at the following address.
If you have a question do not hesitate to contact us via email.
Consult all the articles of LittleBigCode by clicking here: https://medium.com/hub-by-littlebigcode
Follow us on Linkedin & Youtube + https://LittleBigCode.fr/en
References
[1] Fady y. Melhem. Optimization methods and energy management in ”smart grids”. Electric power.
Université Bourgogne Franche-Comté, 2018. English. ffNNT : 2018UBFCA014ff. fftel-01876732f
[2] https://arxiv.org/abs/2012.10504
[3] Filip Tolovski. Advancing renewable electricity consumption with reinforcement learning. Fraunhofer Institute for Telecommunications, HHI 10587 Berlin, Germany.
