Fetch.ai multi-agent system optimization provides real-world benefits to energy sector
The energy sector is set to be one of the biggest beneficiaries of Fetch.ai’s technology. Autonomous agents on the Fetch.ai network have the potential to effect huge change in a marketplace which attracts almost $2 trillion of investment every year. Energy providers and consumers generate data every second, yet the vast majority is inaccessible. Fetch.ai will unlock immense economic potential by making the data available to wider audiences. In this article, we will restrict our focus to the real-world benefits energy companies and market authorities will gain by introducing autonomous agents to the highly competitive marketplace.
In some jurisdictions, deregulation has led to the introduction of private companies and this has had a profound effect on the industry. Traditional, centralized analysis models are now ill-equipped to provide meaningful insights. New models are required, capable of capturing the profit-driven behaviour of self-interested companies.
Fetch.ai machine learning scientist Yujian Ye has collaborated with researchers from Imperial College London to produce two papers. Both papers have been supported by the EU Sysflex project under the European Union’s Horizon 2020 research and innovation program. Both works make significant novel contributions toward enhancing market design and regulation to enable new business solutions. These highlight how Fetch.ai’s autonomous agents will be at the centre of the energy models of the future.
The first paper was published in IEEE Transactions on Smart Grid, an international journal that publishes results of research that relates to, arises from, or deliberately influences energy generation, transmission, distribution and delivery.
It showcases how a new, agent-based and data-driven modelling perspective that utilizes deep reinforcement learning, can enable companies generating electricity to compete more strategically against their competitors.
In markets such as the US, companies place offers for the right to generate electricity. Each company wishes to receive the highest price by the market operator in return for generating the electricity, but each knows their bid will be overlooked if a competitor offers to generate the electricity at a lower price.
There are a complex range of factors that influence how attractive the market operator finds an offer. These factors include the estimated ability of the company in question to generate the electricity it promises. Other factors include the company’s variable costs, the cost of starting up/shutting down the generators and the minimum up/down time it requires. This is in line with the complex bidding structure of many energy markets in the US (e.g. California ISO, PJM, New York ISO).
Traditional analytical market modelling tools neglect such complex tech-economic operating characteristics. Moreover, they make unrealistic assumptions due to the inherent limitations of their modelling capabilities.
The new model outlined in the paper advocates the use of agent-based deep reinforcement learning in multi-dimensional continuous state and action spaces.
Reinforcement learning consists of an agent gradually learning the optimal policy by utilizing experiences acquired from its repeated interactions with the environment, without full system identification nor prior knowledge of the system.
Using this model, companies generating electricity can receive accurate feedback on the impact of their bidding decisions. In turn, this enables them to offer strategically smarter bids than would be possible using current models. This is because an agent-based modelling approach is capable of using historical trading experiences to formulate the optimal strategy. By doing so, a company can gain an advantage over its rivals and is therefore more likely to bid successfully for the right to generate electricity.
The case studies referenced in the paper show that the proposed methodology achieves a significantly higher profit and exhibits a more favourable computational performance than alternative, state-of-the-art methods. Specifically, the proposed methodology achieves 40.56% higher profit over state-of-the-art mathematical programs with equilibrium constraint approach. The proposed methodology also shows a 20% higher profit over Q-learning and 11% higher profit over Deep Q Network approaches.
The second paper, published in IEEE Access, builds upon the work in the first paper. It examines the decision-making process of autonomous agents representing electricity generating companies. It also analyses the market equilibrium brought about by the interactions between these autonomous agents.
The paper Yujian has contributed to proposes a novel multi-agent deep reinforcement learning based methodology in order to achieve this market equilibrium.
Market equilibrium is achieved when none of the competing stakeholders have an incentive to deviate from their strategies. As a result, each stakeholder is prevented from benefiting at the expense of another, and there are necessary compromises across the market. The strategy of each autonomous agent, representing a company, takes into account the market dynamics and potential strategies adopted by its rivals, which are estimated using historical trading data.
These innovative market modelling tools are able to deliver quantitative insights which are crucial for electricity generating companies and other interested parties (e.g. investors in generation technologies). Likewise, the industry regulator (e.g. OFGEM in the UK) benefits from equilibrium analysis as it can use autonomous agents to simulate a range of scenarios and market behaviours before intervening in the market. It also allows the regulator to monitor more effectively the trading of energy, anticipate any shortfall in energy generation and mitigate against the exercise of excessive market power by electricity generating companies.
The AI and machine learning technology behind these case studies is necessarily complex to address the varied bidding approaches adopted by electricity generation companies. If you would like to learn more about the technology discussed in these papers, please email info@fetch.ai. As we have outlined in this article, the papers show how Fetch.ai’s autonomous agents are capable of transforming an industry ripe for change.
