Rock, Paper and Scissors: Which one to go with?
Decision-making can be a daunting task. With so many variables at play, knowing which option is best can be tricky. Choosing between a steak, fish or vegetarian dish at a restaurant can be difficult, deciding which movie to watch on Netflix can be equally tedious, and picking the colour of the t-shirt you would buy for a friend can present you with a similarly hard task! What’s more, these examples do not even contain a high degree of uncertainty or risk. Most real-life decisions are much more complex than this and require us to weigh up countless factors and uncertainties to make a decision that is right for us. Business decision-making is no exception to this rule; however, it can pose even greater challenges due to the large number of people involved and the myriad of factors to be considered. Besides, risks such as financial losses or reputational damage need to be considered carefully when making decisions for businesses as the stakes are usually higher.
Artificial intelligence (AI) is becoming increasingly prevalent in business and has the potential to radically change the way we make decisions. AI gives us access to vast amounts of data that it analyzes and uses to make predictions about different outcomes. In business, the ability to accurately predict the outcomes of decisions is critical to success as managers need to balance risk and opportunity when making important operational or strategic decisions. For example, AI can help manufacturing companies identify potential machine failures before they become an issue and reduce operational downtime as a result. Failure estimations are then used to decide whether to replace machines or repair them to avoid costly delays. This technology, predictive maintenance, allows businesses to proactively manage their resources more efficiently and makes the decision-making process far more effective compared to traditional reactive approaches (e.g., run to failure). Today there is an increasing number of manufacturers adopting predictive maintenance and similar predictive analytics solutions to streamline their day-to-day operational decision-making process and make better informed data-driven decisions. However, it is important to bear in mind that AI will only be helpful to the extent of its accuracy. AI cannot be expected to solve all our problems and make accurate predictions all the time. Like all technologies, it has limitations and needs to be used in conjunction with human judgement and expertise to ensure that it is providing value.
In this blog post, we’ll explore the nature of decision-making under uncertainty and the impact of imperfect forecasts produced by AI. Moreover, the blog incorporates a mini-game — Rock, Paper and Scissors *— which aims to illustrate the concept of decision-making under uncertainty and how helpful the estimations (possibly provided by an AI) would be to overcome this inherent challenge. Even better, you can experience the mini-game yourself!
It’s almost impossible to develop a consistently winning strategy in this game because the outcome is dependent on chance and cannot be predicted in advance. However, let’s imagine for a second that we could develop a way to predict the opponent’s move, using some imaginary AI, before we started playing the game. Then, every time we played a round of RPS, we could work out which of the three shapes we should play to win that particular round. Nevertheless, it’s still possible that our opponent might choose to play a different shape than AI have predicted, in which case we might lose that particular round. After all, AI is never 100% accurate.
As part of this blog post, I programmed a mini-game of RPS to help illustrate this concept (using Streamlit — an open-source framework that lets you turn Python scripts into shareable web apps in minutes). You can find links to the game as well as the source code at the end.
In the single-play mode, the game allows you (Player 1) to get estimates about the possible move of your opponent (Player 2) in the next round before the round starts. In this way, you can pick the winning move and hope to come out on top. The estimates are generated randomly however, an estimation accuracy that should be set by you at the beginning of the game is taken into account (the aim is to mimic an imperfect AI-powered predictive model). You can vary this percentage between 0% and 100% (the default setting is 70%). Eventually, the estimate could or not be correct on a single instance (unless you set the estimation accuracy to 100%) and relying on the estimate provided could let you either win or lose on the round (see below example which demonstrates a loss due to false estimation).
You can try it yourself. Go to the game, select ‘Single Play’ on the menu left-hand side, and follow the instructions to play.
Losing a game based on a false estimation does not mean estimations are useless. In the long run, the rate of your wins is expected to converge to the estimation accuracy that has been set at the beginning and you may anticipate dominating the opponent as the rounds continue (unless you set the estimation accuracy below 50%). For example, after the initial 10 rounds you may still not be dominating the game despite 70% estimate accuracy (e.g., 4 wins, 4 losses, 2 draws — see below example).
However, as the game progresses and the number of rounds increases, you’re bound to get more wins and your rate of wins should eventually become closer to the estimation accuracy that has been set at the beginning (see below example with 1000 rounds).
Go ahead and try this yourself; select ‘Simulation’ on the game menu and run a simulation. You can set the number of rounds (up to 1000). You will notice that the more rounds you play, the closer your winning rate will get to the estimation accuracy (unless you pick a strategy other than “rely on the estimation”). If you want to experiment with different outcomes, you can pick different strategies: Always rock, always paper, always scissors or random.
Essentially, if the estimations you get are proven to be reasonably reliable and you are consistent with a reasonable decision strategy (i.e., rely on the estimate and play accordingly), you can expect the strategy to pay off in the long run.
The nature of decision-making under uncertainty implies that we are never completely certain about the results of our decisions (even if there are forecasts available) and there is a certain amount of risk involved in making choices. Although AI would be helpful in most cases, we should not blame AI if the outcome of our decision that takes AI’s forecasts into account turns out to be negative on a single occasion.
For instance, a predictive maintenance solution may mislead maintenance engineers with a false positive failure alarm (i.e., an inaccurate alarm when there is actually no failure risk) or a false negative inaction (i.e., no alarm despite an actual failure risk). Engineers have to make consistent decisions about their maintenance plans based on a determined decision strategy. This strategy should also incorporate a policy about how to utilize AI’s inputs in any individual case. Occasionally, they may end up halting the production mistakenly to inspect a machine that AI pointed out with an alarm about a potential breakdown (or vice versa — miss a failure due to no alarm and deal with a production stoppage); similar to making a false “rock” move due to the expectation of a “scissors” move from the opponent while the actual move of the opponent is “paper”. However, single instances do not indicate that AI is less valuable. As the number of cases goes up, people who invest in AI should expect their consistency in a decision strategy to pay off with the help of reliable AI.
*Rock, paper and scissors (RPS) is a simple hand game that is played by two people. The game is often used as a way to make decisions or settle disputes. In the game, each player simultaneously forms one of three shapes with an outstretched hand. These shapes are “rock” (a closed fist), “paper” (a flat hand), and “scissors” (a fist with the index and middle fingers extended, forming a V). The winner of the game is determined by the rules of the game, which state that rock beats scissors, scissors beats paper, and paper beats rock. The game is often played as a series of rounds, with the winner of each round being the player who has played the shape that beats their opponent’s shape.
Link to the game: https://iozpoyraz-rockpapersciss-game.streamlit.app/
Link to the game source code: https://github.com/icanpoyraz/rockpaperscissors
