Simple Correction Improves Computational Materials Discovery
By Addressing the Root Cause of a Simulation Calibration Error, New Materials Discovered With AI Will Be More Relevant
TRI’s Accelerated Materials Design and Discovery team has released a paper that has strong implications for anyone using computational materials science and artificial intelligence for materials discovery. The new paper highlights a problem that is subtly embedded inside nearly all campaigns to identify new candidate materials via simulation. The problem lies in a common correction that is meant to better align simulation with reality, but ironically, this correction can lead to unphysical results. Fortunately, the team proposes a simple solution.
TRI’s VP of Energy & Materials, Brian Storey, speaks with Brian Rohr, CEO of Modelyst and lead author of the study (and former intern at TRI in 2018), to explain the problem and solution. Modelyst has been working closely with TRI and its partner university labs to develop custom data infrastructure and machine learning tools for the past three years. While working closely with the datasets that TRI uses for machine learning studies, Dr. Rohr and the TRI team noticed some unusual results within these discovery campaigns, which led to the team digging deeper. Let’s let Dr. Rohr explain.
Our correction makes the prediction of new materials more physically accurate and shows that many computer-reported stable materials are not discoveries at all.
STOREY: There have been recent news reports highlighting the use of AI to discover new materials for clean energy technologies. Can you explain to our readers what this work is all about?
ROHR: There are unlimited possibilities for the materials we could theoretically make, and some of these unknown materials could have game-changing properties and behaviors. In principle, we can take any number of elements from the periodic table in any proportion and see if those elements can form a new material. There are infinite possibilities to try, but we are only interested in “stable materials.” If a material is unstable, kind of like a mixture of oil and water, it will quickly separate into its components. However, if a material is stable, it will last for a long time. We would want to use these new materials in real devices like batteries and solar cells, so only the long-lasting, stable ones are interesting.
The brute force discovery method would be to go into the lab and keep mixing together more and more combinations of different elements in different amounts, and occasionally, we would find a new stable one. However, that would be impractically slow since it is extremely time-consuming and requires deep expertise to study even one of these possible new materials in the lab, and there are infinite possibilities.
STOREY: So how could we try to find these stable materials faster?
ROHR: Over the past 10–15 years, there has been a focused effort to use computational methods to simulate lots and lots of possible materials on the computer. Using a computational method called Density Functional Theory (DFT), we can predict the stability of a new material on the computer without making it in the lab. The hope is to use these computer simulations to narrow down the possibilities to the most promising candidate materials and try to make just those in the lab.
Even though DFT was developed in the 1960s, this approach has become more common recently because computers are so much more powerful. We can run more simulations than ever before, and we can use AI to intelligently select which simulations to run. In fact, when I was an intern at TRI, some of the researchers were working on AI-driven materials search.
STOREY: So, can we trust a computer’s prediction of the stability of a new material?
ROHR: Not directly. Just like you need to tune your piano, you need to calibrate the computational results.
STOREY: What is the new thing you discovered about how to calibrate DFT’s predictions, and how is it different from the conventional approach?
ROHR: Let me explain with an analogy — which, of course, is just for illustration. Imagine that we suspect our car’s speedometer is not accurate. So we go out on the highway and test our car against a radar gun that we know is good. We plot several data points of speedometer speed versus radar gun speed and get a plot that looks like the figure below, which confirms our suspicion. The speedometer is inaccurate.
One simple fix would be to look at our data and note that on the highway, our speedometer is off by about 10 mph, so we could simply subtract 10 from the speedometer. This correction would shift the data points down to lie along the black line — where the two measurements would agree. But this fix ignores that our error could be one of calibration. A speedometer works by measuring the rotation rate of the wheel and then multiplying that by the tire size to get the speed of the car. If we have the tire size off by a little, we’ll get the speed wrong. With the wrong tire size in the calibration, the speedometer would be correct at 0 mph, but the error grows with the speed.
If we suspect our speedometer is not calibrated well, we wouldn’t just measure at a highway speed. We check across a range of speeds and start to see a clear linear trend emerge as shown in the figure below. From this data it is clear that adjusting the slope of the data would be a better approach than just subtracting a constant at highway speeds.
STOREY: So what does this have to do with materials discovery?
ROHR: The conventional correction in DFT is similar to this analogy. Materials simulation data is commonly corrected to match experiments by introducing a constant offset, but a different constant is applied to each element. To extend the analogy, this would be like applying a 10 mph constant correction to sports cars (because they drive fast on the highway) and a 5 mph constant correction to slower-driving minivans.
In materials, the reactive elements, like fluorine (F), have bigger predicted stabilities and bigger errors (like our sports car that tends to go fast). The less reactive elements, like antimony (Sb), have smaller predicted stabilities and smaller errors (like our minivan going slow). The common DFT correction method for material stability prediction involves subtracting “ten” for the reactive elements and “five” for the less reactive ones.
All we did was go back to the drawing board and fit a simple line to the calibration data that is good over all speeds. Our correction is a single parameter to re-calibrate all materials rather than apply different corrections for different elements.
STOREY: That’s it?
ROHR: If you remember your algebra, you might remember the equation of a line is y=mx + b. Traditionally, the DFT community has changed b. We just think you should change m. That is really all there is to it.
STOREY: So what is the impact of this simple proposed DFT correction?
ROHR: It turns out our change starts to matter as these DFT databases get bigger. Until very recently, most of the studies have been focused on materials far away from the “reference state,” loosely analogous to speedometers far away from 0 mph. Essentially, we’ve only been looking at the speedometer of sports cars on the highway and minivans on the local roads, but neither car in the parking lot, so the problem wasn’t so noticeable.
However, as the material simulation databases grow in size and complexity, we see more materials that are closer to the “reference state” of 0 mph. That’s where the constant offset correction gives some really weird results. By analogy, it’s like starting to look at your car’s speedometer when you’re going 3 mph in a parking lot, and it reads -7 mph. All of a sudden, it’s obvious that something is wrong.
Our proposal is rather than compensate for the error in each car, we address the root cause of the calibration error and simply adjust the slope of all the data. This correction works not only to match the experiment to the simulation, but importantly, our correction has the right behavior in the parking lot.
In materials simulation with the traditional offset correction, DFT will incorrectly predict some materials to be stable (analogous to the speedometer saying the car is going backward). With our correction, it becomes clear that these materials would fall apart right away. So, our correction makes the prediction of new materials more physically accurate and shows that many computer-reported stable materials are not discoveries at all.
These may look like simple corrections in a simulation database, but as researchers attempt to make some predicted stable materials in these databases, these prediction errors can result in significant costs and wasted resources, including time.
STOREY: Does this mean existing DFT databases aren’t useful?
ROHR: No, they’re still very valuable resources. The existing simulation work is still 100% valid. It is only the correction that needs to change, which is added in after all the difficult and time-consuming work. We would advocate that all DFT databases publish the original uncorrected data and let the users select their calibration approach. Our correction approach is very simple to implement, and that is the beauty of it.
