Using AI for managing renewable energy generation and management

Abhijit Gupta
Intel Student Ambassadors
4 min readMar 30, 2020


Harnessing power of AI to build a robust solar irradiation forecasting pipeline for efficient management and integration of solar energy in power grids.

The demand for clean energy is rapidly increasing. The current pace of technological development makes it commercially viable to harness energy from sun, wind, geothermal and many other renewable sources. As renewable power plants continue to expand, it will also be necessary to determine their optimal sizes, locations and configurations. The energy/power output of these plants is defined by the environmental factors such as wind speed, the intensity of solar radiation, cloud cover and other factors. This in turn necessitates the prediction of these environmental factors such as wind speed, direction and solar radiation in the region of the power plant. There is a critical need of gaining real time high-fidelity observability, control, and improving renewable generation forecast accuracy to enhance the resiliency and to keep the operational costs sustainable. The above-mentioned challenges have motivated us to employ machine learning techniques to support better management of renewable energy generation and consumption. With about 300 clear and sunny days in a year, the calculated solar energy incidence on India’s land area is about 5000 trillion kilowatt-hours (kWh) per year (or 5 EWh/yr). The solar energy available in a single year exceeds the possible energy output of all the fossil fuel energy reserves in India. Specifically, we intend to build an end-to-end pipeline for multi-time horizon forecasting for estimation of short and long-term solar radiation forecasting using machine learning approach. This would be beneficial as one of the low-cost methods for efficiently integrating solar energy into the grid. Here’s a brief outline of our approach:

Methodology / Approach

We’ll be procuring raw data from Solar Resource Assessment based on satellite imagery (SEC- NREL Project), Indian Solar resource Map, and SURFRAD (Surface Radiation Budget) Network, which provides reliable satellite-based estimates of solar radiation throughout the day. After cleaning and munging data, followed by exploratory data analysis, we will construct an RNN model using Intel’s Tensorflow and Intel Distribution for Python and PyDAAL, which provides fast vectorized implementation of different approaches used in data analysis. Capability to predict for multi-time-horizons (1-hour, 2 hour, 3 hour and 4 hour period) will make our proposed method very relevant for industry applications. The real-time data can be fed to the RNN model/machine learning model and owing to its lower forward inference time, predictions can be made for multiple time horizons. Our long-term goal is to be able to work on multiple renewable sources and coming up with an approach that would facilitate the formulation of well-informed energy policies, for example by helping to estimate important parameters such as the appropriate spinning reserve levels and storage requirements.

Technologies Used

Our framework pipeline will make use of TensorFlow with Intel® MKL DNN for CNN. It will be tweaked to maximize performance. For pre-processing the data, we will be using Intel pyDAAL framework. Other tools that we intend to use are Intel TBB (Threading building blocks) with Dask.

Data Sources

Solar irradiance distribution
Solar irradiance distribution
Solar irradiance distribution
Solar irradiance distribution

Repository Link:

Video Description:

Model Description

Given data going as far back as lookback timesteps (a timestep is 10 minutes) and sampled every steps timesteps, can we predict the solar irradiation in delay timesteps?

•Lookback = 1440 — Observations will go back 10 days

•Steps = 2 — Observations will be sampled at three data point per hour(20 min recording)

•Delay = 144 — Targets will be 24 hours in the future

Model architecture
Time Series description


Data description

Explained variance score: 0.9897

r2 score: 0.9895

Mean Absolute Error(5-fold cv): 10.04±3.01

Root Mean Squared Error(5-fold cv): 14.44±2.5

Test Mean Absolute Error: 11.38 [W/m**2]

Test Root Mean Squared Error: 15.60 [W/m**2]

Prediction for 1000 time steps
Prediction for 2000 time steps
Long Term prediction [24h period]