Valuing AI — Part 3: Calculate your risks!
“Prophecy is a good line of business, but it is full of risks.”
— Mark Twain
Artificial Intelligence (AI) is highly popular with its almost accurate predictions. Probably for that very reason, Facebook (or should I call it ‘Meta’ to be trendy) named its open-source forecasting package, a procedure implemented in R and Python programming languages, as Prophet. It is indisputable that AI makes no pretensions to prophesy or anything transcendental, dismissing the fact that an engineer created an artificial messiah named ‘A.I. Jesus’ which is trained in biblical scripture and has been coming up with its own prophecies” (Sourced from The Next Web). AI is just math and nothing more than a bunch of lines of code. But thanks to all those smart enough engineers and scientists in the AI community it thrives in making predictions where it makes sense and is possible. Nevertheless, it has its own risks and costs, as highlighted by Mark Twain.
Those risks impede many decision-makers to trust AI and give it a seat. Although being risk-averse is rational and necessary to dissociate being a manager versus a gambler, every rational decision-maker knows that risks cannot be canceled out. As long as they are calculated and justified risks are an undeniable element of business decision-making (as a matter of fact, this probably applies to all decision-making activities). Having said that, how could one measure the risk of an AI solution and associate it with its value (i.e., considering the risks how much one should pay for the solution)?
In this three-part blog series, I bring a few concepts up for discussion that would help establish a framework for translating AI’s value. These are:
- Expected Value of Perfect Information
- Bayesian Thinking
- Value at Risk
In the first post, I presented Expected Value of Perfect Information and illustrated a sample use of it on the Predictive Maintenance use case, an application of AI in manufacturing (Part-1) whereas in the second one, Bayesian thinking was discussed, again with an example referring to the same use case (Part-2). In this last piece, I address the risk aspect of value and bring forward the concept of Value at Risk (VaR) which is a well-known and widely applied method by finance people.
According to its formal definition, VaR is a statistic that quantifies the extent of possible financial losses within a firm, portfolio, or position over a specific time frame; this metric is most commonly used by investment and commercial banks to determine the extent and probabilities of potential losses in their institutional portfolios (Investopedia definition). More simply, it is a measure of the risk of loss for investments (Wikipedia definition). VaR emerged as a distinct concept after the stock market crash of 1987, the so-called Black Monday (Wikipedia definition) and it is now a natural selection in the mainstream financial risk management practice.
Here is a solid example: For example, if a 10-day VaR is stated as $657,942 to a 95% level of confidence, this means that during the next 10-day there is only a 5% chance that the loss the next day will be greater than $657,942 (Sourced from YieldCurve.com and cQuant.io). See the below figure for the visualized explanation:
Well, that’s a nice piece of information but how this concept would be helpful in terms of valuing AI by addressing risks? Let me use an example to make my case.
How about that? Consider the purchase of a Predictive Maintenance solution as a risky asset investment just like investing in a company’s stock listed in NYSE (no prediction model is perfect, there are always erroneous predictions; therefore predictive solutions are risky). In this context, where do you think risks would stem from? The answer is any breakdown that the prediction model failed to warn. Companies purchase (invest in) Predictive Maintenance solutions in order to get alerts of anticipated machine breakdowns. The Predictive Maintenance investment yields high returns once the company’s maintenance team reacts to an alert and prevents a failure as well as a consequent costly downtime. On the contrary, their investment loses value if it fails to predict some of or in the worst case all of the breakdowns.
From this point of view, losses of the Predictive Maintenance investment can be calculated including but not limited to the following elements: (i) cost of a breakdown in terms of fixing workforce and spare parts, (ii) cost of downtime or lost production, and (iii) bringing the economic lifetime of the machine down. On the other hand, the confidence level (e.g., 95%, 90%) can be based using the model’s track record at other sites and applications.
Proposing procedures and formulations for calculating a Predictive Maintenance solution’s VaR is beyond the purpose of this blog post, however, we can at least name the generic methods proposed by the financial risk management discipline for calculating the VaR. These are the historical method, The Variance-Covariance Method, and the Monte Carlo simulations (Sourced from investopedia.com).
Calculating your risks is essential to deal with an unknown known type of uncertainty. It also helps optimize a decision maker’s attitude towards risky investments; prevents being not only over-conservative but also over-radical. I believe it is crucial to have such an analytical mindset while valuing technologies such as AI that have a valuation process as complex as risky financial assets.
The legendary investor Warren Buffet once said: “Risk comes from not knowing what you’re doing”. VaR could offer a procedure to know what your AI solution is actually doing.
İhsancan Özpoyraz | Senior Consultant, KoçDigital
