Geothermal Plant Source

Digital Twin Design for Renewable Energy Exploration

Building Digital Twins with XGBoost and Developing Trust in Them through Explainable AI

Digital twins are virtual representations of physical objects or systems. These representations allow us to monitor the digital twin when exposed to external/internal variables and predict how it will perform in a production environment.

One of the most complex mechanical environments can be found in high-temperature, high-pressure drilling, like in the geothermal energy industry. As a critical component of a transition toward cleaner renewable energy sources, finding more efficient methods for exploiting viable thermal reservoirs is essential. As drills penetrate deeper into the earth, the pressures and temperatures push drilling equipment to its limits, and the rate of penetration (ROP) can drop to less than a few feet per minute. In this tutorial, we will leverage the Digital Twin AI Reference Kit to build a digital twin for a geothermal drilling rig to simulate the impact on ROP of manipulating four drilling control parameters: weight on bit (WOB), rotary speed (RPM), flow rate (FLOW), and torque (TOR).

For real-time drilling simulations, we have strict inference requirements for our model, especially if we intend to partner this digital twin with a subsequent optimization algorithm. To meet our inference speed requirements, we will leverage Intel optimizations for XGBoost from the daal4py module (Figure 1).

Figure 1. Inference on an Intel-optimized XGB Regressor model and daal4py model conversion inference relative to stock XGB Regressor based on the MOSFET use-case in the Digital Twin AI Reference Kit. We include a comparison against an older version of XGBoost (0.81) before our accelerations were upstreamed into the stock package. Please remember that the performance benefit will result from both Intel Optimizations and version updates. On the other hand, daal4py gain over XGBoost v1.5.0 is purely due to Intel optimizations as part of the oneDAL library. Source

The following .yml file can be used to replicate my conda environment:

Geothermal Drilling Challenges

The expectation of this tutorial is not to become a drilling expert but to learn how to identify a physical system and decompose it into its inputs and outputs to build an AI digital twin model. Still, it’s essential as data scientists that we understand the problem domain.

Geothermal energy exploration aims to discover and exploit the earth’s natural heat as an energy resource (Figure 2). Geoscientists study the subsurface to find in-field, near-field, and deep geothermal systems with enough heat to justify the cost of drilling wells and building a power plant. The focus of our digital twin will be to understand the performance of wells that need to be drilled deep enough to access heat for electricity production.

Figure 2. The diversity of geothermal resources and applications, delineated within three resource categories: geothermal heat pump, hydrothermal, and enhanced geothermal systems source: 2-GeoVision-Chap2-opt.pdf

Drilling in high-pressure, high-temperature environments presents unique challenges because they require specialized tools that can withstand these extremes. We need temperatures of at least 350 degrees F for flash and dry steam geothermal powerplants (Figure 3), and that kind of heat is rarely available above 10,000ft (1.89 miles). This pressure and temperature wreak havoc on sensors and require specialized bits, casing, and drilling fluids.

Figure 3. The continuum of geothermal energy technology applications and uses Source

Control parameters such as WOB, RPM, FLOW, and TOR are used to optimize drilling conditions:

• WOB: the amount of downward force exerted on the drill bit
• RPM: the number of complete rotations made by the drill bit
• FLOW: the volume of liquid or slurry that moves through a pipe in one unit of time
• TOR: the rotational force between the drill string and the rock

Processing and Analyzing our Geothermal Drilling Dataset

We will use the Electronic Drilling Recorder (EDR) data from the UTAH FORGE geothermal project to train our drilling rig. Utah FORGE is a dedicated underground field laboratory for developing, testing, and accelerating breakthroughs in enhanced geothermal systems technologies (Figure 4).

Figure 4. Image of Utah FORGE Well

Our data is associated with the drilling of the Utah FORGE 58–32 MU-ESW1 well near Roosevelt Hot Springs. It is a one-foot-interval drilling dataset for well 58–32 that covers from 85.18 feet to 7536.25 feet in depth. It contains information such as depth, rate of penetration, weight on bit, hookload, temperatures in and out, pump pressure, flows in and out, H2S, etc. (Figure 5).

Figure 5. Preprocessed geothermal drilling data from left to right: ROP, WOB, Temp Out, Temp In, Pit Total, Pump Press, Hookload, Surface Torque, Rotary Speed, Flow In, and Flow Out.

Basic Data Processing

For digital twins, we must first define the causal relationships within the modeled system. To achieve this, we will separate the inputs and outputs of the drill rig. This leaves WOB, TOR, RPM, and FLOW as inputs and ROP as our sole output (Figure 6). Lastly, we will remove the first 1000ft of data to focus on areas where drilling is slow and can be improved, like zones of high pressure and temperature.

Figure 6. Plots of our processed training data (left to right): ROP, WOB, TOR, RPM, and FLOW.

Statistical Feature Analysis

First, we will use a Pearson correlation matrix to analyze the relationships between our features (Figure 7). We see the following relationships in the correlation matrix:

• Flow in, rotary speed, and ROP show moderate positive correlations. This makes sense because fluid flow through the drill string helps spin the bit, leading to higher rotation speeds and rates of penetration.
• WOB shares the highest correlation with ROP, which also makes sense because ROP defines the amount of downward force applied to rock during drilling.

Figure 7. Pearson correlation matrix of geothermal drilling data created using pandas and seaborn

Next, we will perform a pair plot analysis, which uses a “small multiple” approach to visualize the univariate distribution of all variables in a dataset along with all of their pairwise relationships (Figure 8). From the diagonal histograms, we see various complex multi-modal and skewed distributions.

Figure 8. Pair plot showing with cross-plotted values for all model features and histograms along the diagonal

The kernel density maps associated with depth (Figure 9) indicate a complex distribution of clusters for each control parameter. This is representative of multiple rock types and the parameters used to drill through them. In the future, we should include rock property logs to give our model more lithological context.

Figure 9. Depth kernel density maps of depth and all other models features

Lastly, we can use geothermal_data_final.describe() to generate a statistical profile of our data (Figure 10).

Figure 10. Statistical summary of our training dataset

Since RMSE is essentially standard deviation plus the random and systemic errors of our data and model, it is a good goal to achieve an RMSE far below our ROP standard deviation (23.36 feet/hour).

Building our Digital Twin — daal4py and XGBoost Optimized Training and Inference

We will start by training a linear regression model (Figure 11) and evaluating it to establish an RMSE reference point and ensure that our XGBoost model will outperform a simple linear regressor.

Figure 11. Code snippet of linear regression baseline

We achieve an RMSE of 18.39 feet/hour. This is just 4.97 feet/hour less than the standard deviation of our training data ROP.

===== Running Benchmarks for Linear Regression =====
Training time =  0.0015287399291992188
Prediction time =  0.0011591911315917969
MSE: 338.375
Root MSE: 18.394972139147153

Let’s see if we can do better with XGBoost. We implement XGBoost and automated hyperparameter optimization using the GridSearchCV and xgb methods (Figure 12). Line 21 is where we use the XGB_prediction _daal4py custom function from the digital-twin AI Reference Kit to convert our XGBoost model to daal4py.

Figure 12. Code snippet with XGBoost training and daal4py inference

===== Running Benchmarks for XGB Hyperparameter Training =====
Fitting 4 folds for each of 8 candidates, totaling 32 fits
Training time = 15.96353268623352
Prediction time =  0.027706146240234375
daal4py Prediction time =  0.003979682922363281
Root MSE:  6.463745044476925
daal4py Root MSE:  6.463745044476925

We did it! With the hyperparameter-optimized XGBoost, we achieved an RMSE of 6.46 feet/hour, which is 16.9 feet/hour lower than the standard deviation of our ROP. It also performs 64.9% better than our linear regression baseline. It is also worth mentioning that inference using daal4py gave ~86% speedup over the XGBoost model, a significant performance improvement.

Evaluation with Synthetic Testing Data

Now that we have a satisfactory digital twin of our drill rig, we can perform a quick test by giving it some values from a synthetically generated well (Figure 13). Our digital twin achieves an RMSE of 10.85 feet/hour. This is more than satisfactory for this article’s purpose.

Figure 13. Line plot of Digital Twin Simulated ROP (Blue) and Synthetic Well ROP (Orange)

Building Trust in Our Twin — Explainable Artificial Intelligence (XAI)

Now that we have built our digital twin, XAI can help us understand the learned relationships between the input and outputs of our model. We will explore a particular XAI implementation using the open-source SHAP library. The core idea behind the “Shapley Value-Based Explanations” is to use cooperative game theory to allocate credit for a model’s output f(x) among its input features. This is achieved by measuring the impact of features on model output based on when they are present, missing, or in combination with other features.

We will use the SHAP library’s waterfall visualizations to show how each feature pushes the model output from the base value (the average model output over the training dataset) to the model output. Features pushing the prediction higher are shown in red, and those pushing the prediction lower are in blue. When ROP = 16.301 feet/hour, this is the impact of each feature (Figure 14):

• Flow In (GPM) brings the model down from the base value by -4.93 ROP units.
• WOB (k-lbs) brings the model down from the base value by -3.49 ROP units.
• Surface Torque (psi) brings the model down from the base value by -0.93 ROP units.
• Rotary Speed (rpm) brings the model up from the base value by +0.29 ROP units.

Figure 14. SHAP Waterfall plot showing the contributions of all model input features for a sample resulting in ROP 16.30 feet/hour

When ROP = 35.647 feet/hour, this is the impact of each feature (Figure 15):

• Flow In (GPM) brings the model up from the base value by -11.91 ROP units.
• WOB (k-lbs) brings the model down from the base value by -1.55 ROP units
• Surface Torque (psi) brings the model down from the base value by -0.35 ROP units.
• Rotary Speed (rpm) brings the model up from the base value by +0.27 ROP units.

Figure 15. SHAP Waterfall plot showing the contributions of all model input features for a sample resulting in ROP 35.47 feet/hour

This analysis is vital to understanding how our model reached a specific result. We can use it to guide data cleaning, processing, and feature engineering in future iterations if we identify problems or advantages with particular features.

The SHAP Heatmap (Figure 16) serves a similar purpose as the waterfall plot. However, instead of looking at the impact sample by sample, it shows the high-level impact of all features across our model’s outputs. Again, this visualization can help us understand how different features impact our model. In the case of the sharp increase in ROP around instance (sample) 1750, we see very high SHAP values for both Flow In and WOB, indicating that these two features are primarily responsible for the resulting increase in ROP.

Figure 16. SHAP heatmap showing the SHAP values for all outputs of our model

XAI is an active area of research. One of the significant challenges with XAI tools like SHAP is that they are difficult to scale due to their computational demand. With just a few million rows, SHAP could quickly become a non-starter due to the compute times for the explainers and SHAP value calculation. One solution could be to reduce our dataset, which would risk losing critical insights. Another solution is to try parallelization.

Parallelizing SHAP calculations can improve performance by running computations across all CPUs in our cluster. Databricks has done some work on parallelizing SHAP (Figure 17). Intel also provides a Spark Tuning Guide.

Figure 17. Single-node vs Parallel SHAP Calculation Execution Time (1M rows, 10 columns) Source: Databricks

Summary and Discussion

We explored how to use Intel-optimized XGBoost and daal4py libraries to train an machine learning digital twin for ROP simulations. We took this a step further by implementing an XAI workflow with SHAP to build trust in our model by interpreting the impact of input features on the simulation outputs of our digital twin.

As a problem like this scales, it is important to consider the compute challenges that might arise, so using hardware-optimized libraries for training/inference and parallelized processing for SHAP could be what meets the feasibility requirements of this problem.

To build a complete solution for geothermal drilling optimization, there are a few more considerations that we should make as part of our future work.

• To turn our digital twin workflow into an optimization engine, we could pair it with a Bayesian optimization algorithm, reinforcement learning, etc.
• We should include rock property logs to give our model lithological context.
• For this digital twin to be viable in the real world, we would need much more data, but our workflow would likely remain the same.
• To build a more complete digital twin, we should consider things like bit wear and choke valve controls.

Lastly, it is important to consider that the framework offered by the Digital Twin AI Reference Kit could also be used to model other complex systems.

Don’t forget to follow my profile for more articles like this!

Sources and References

• Utah FORGE | Energy and Geoscience Institute at the University of Utah. (2018). Utah FORGE: Logs and Data from Deep Well 58–32 (MU-ESW1) [data set]. Retrieved from https://dx.doi.org/10.15121/1452735.
• NeurIPS | A Unified Approach to Interpreting Model Predictions
• Geothermal Technologies Office | GeoVision: Harnessing the Heat Beneath Our Feet
• Intel | Building a MOSFET Digital Twin for Design Exploration: Modeling Sub-threshold Voltage Leakage Current using XGBoostRegressor