Writing an ‘Akinator Game’-inspired app to diagnose Failure Modes on industrial equipment
When I was hired as a Maintenance Supervisor in a manufacturing industry, I was challenged to develop an application that could diagnose the failure mode in the installed equipment. The goal was that, whether peers had maintenance training or not, they could know what the specific problem was, in the famous style of the internet game ‘Akinator’. Knowing this, they would quickly know how to solve it without the need for an expert.
The motive of this article is to describe the three steps necesarry to develope an ‘Akinator Game’-inspired app to diagnose Failure Modes on industrial equipment, which are:
- Learning about the Akinator algorithm
- Obtaining the equipment data
- Implementing my dataset in the algorithm
Learning About the Akinator algorithm.
After an exhaustive google lookup, we can find out that the Akinator algorithm is based in what is called a Classification and Regression Tree (CART). To make one, we need to create a root node. This will receive an entire data set containing the classification to give a predictive output in the last column, and the characteristics of each type on the other ones. An example is shown below.
Now each node asks a true-false question about one of the features. In response to this question, we split, or partition, the data into two subsets. These subsets then become the input to two child nodes we add to the tree.
The goal of the question is to unmix the labels as we proceed down. In other words, to produce the purest possible distribution of the labels at each node. For example, the input to this node contains only a single type of label. We’d say it’s perfectly unmixed; there’s no uncertainty about the type of label.
On the other hand, the labels in this node are still mixed up, so we’d ask another question to further narrow it down.
The trick to build an effective tree is to understand which questions to ask and when; to do that, we need to quantify how much a question helps to unmix the labels. We can quantify the amount of uncertainty at a single node using a metric called Gini impurity.
And we can quantify how much a question reduces that uncertainty using a concept called information gain.
We use these to select the best question to ask at each point and use it to recursively build the tree on each of the new nodes. We continue dividing the data until there are no further questions to ask, until we add a leaf.
Let’s work on impurity. This is a metric that ranges between 0 and 1 where lower values indicate less uncertainty, or mixing, at a node. It quantifies our chance of being mistaken if we randomly assign a label from a set (say ‘Apple’, ‘Lemon’ or ‘Grape’ for the given example) to an observation in that same set.
After the impurity has been worked on, let’s move to information gain. Information gain let us find the question that reduces our uncertainty the most. This it’s just a number that describes how much a question helps to unmix the labels at a node. Here’s the idea: we begin by calculating the uncertainty of our starting set, then, for each question we can ask, we’ll try partitioning the data and calculating the uncertainty of the child nodes that result. We’ll take a weighted average of their uncertainty because is better to have a large set with low uncertainty than a small set with high. Subsequently, we’ll subtract this from our starting uncertainty. And that’s our information gain. As we go on, we’ll keep track of the question that produces the most gain; that will be the best one to ask at this node.
With these concepts in hand, we’re ready to build the tree. And to put this all together we’re gona walk through the algorithm as it builds a tree for our training data: First we call our tree function and receives the entire dataset, with this info the object node calculate the gini impurity of the set and determine the best question to ask using the information gain. Using this question we then split our data set and calculate the information gained for each one. As we go on, we must keep track of the question that produces the most gain and recursively split the dataset, until there’s no more questions to ask, so the information gain will be zero, thus, this node becomes a leaf that can predict the classification with some percentage of confidence.
For deeper explanation on CART algorithms, please check Josh Gordon’s Machine Learning Recipes youtube channel in the link. You can get the code from the example above in Josh Gordon’s Github repository.
Obtaining the Equipment data.
In the industry there is a document that describes each failure mode that a piece of equipment can present, the Failure Mode and Effects Analysis (FMEA). It is considered a living document since each new failure mode is documented and updated. From reading the document we can obtain a dataset that have a Failure Mode in one column and effects in the others. Since this is a private document from the company, we will use an example for an industrial sealer dispenser that looks like the following using a binary encoding if the effect is present in the failure mode:
You can download the csv file from my Github repository here. Another way to do this is to create a dataset based on the expertisse of the team members, this could add some reliability, and even better if we combine FMEA, team experience and equipment manuals. Nevertheless, since this is a very extense process because we have to evaluate each category step to step, for the case of this studie we’ll use the Seal System Failure Mode Dataset above.
Implementing my dataset in the algorithm
For the last part, we use Pandas python’s library to read my CSV data to train the model in Josh Gordon’s code to create a Classification Tree as shown below.
import pandas as pd
training_data = pd.read_csv('Seal_DB.csv')
# Change the columns position to have the Failures in the last
temp_cols = training_data.columns.tolist()
new_cols = temp_cols[1:] + temp_cols[0:1]
training_data = training_data[new_cols]
# Column labels.
# These are used only to print the tree.
header = training_data.columnsIndex(['No application of material',
'Too little or too much material is applied',
'No application of material in places', 'Material is escaping',
'Air is escaping from pneumatic drive',
'AK 4xx installation gun/gun is not opening',
'No supply of material from dosing unit/barrel pump',
'Material is not sufficiently heated',
'Insufficient supply of material',
'Material contains hardened pieces (lumps)',
'AK 4xx installation gun/gun is not closing properly',
'Material is escaping from material unit',
'SCA controller switched off',
'Driver card in the controller is defective', 'Valve bolt is jammed',
'Dosing unit is filling up', 'Barrel pump compressed air supply off',
'Stop valve closed', 'Barrel empy', 'Heater switched off',
'Heating cartridge is faulty', 'Temperature sensor is faulty',
'Dosing unit is leaky', 'Material hose is blocked',
'Material hose is leaky', 'Material unit is blocked',
'Nozzle is blocked', 'Air in the system', 'Lumps on the pintle seat',
'Pintle seat damage', 'FAILURE MODE'],
dtype='object')Finally, we run the code to create the Tree and predictions:
my_tree = build_tree(training_data)
print_tree(my_tree)Is Pintle seat damage >= 1?
--> True:
Predict {'AK 4xx installation gun/gun is not closing properly because of lumps on the pintle seat': 1}
--> False:
Is Lumps on the pintle seat >= 1?
--> True:
Predict {'Material contains lumps because of air in the system': 1}
--> False:
Is Too little or too much material is applied >= 1?
--> True:
Is Material unit is blocked >= 1?
--> True:
Predict {'Insufficient supply of material because the material hose is leaky': 1}
--> False:
Is Insufficient supply of material >= 1?
--> True:
Is Material hose is leaky >= 1?
--> True:
Predict {'Insufficient supply of material because the material hose is blocked': 1}
--> False:
Predict {'Insufficient supply of material because the dosing unit is leaky': 1}
--> False:
Is Dosing unit is leaky >= 1?
--> True:
Predict {'Material is not sufficiently heated because the temperature sensor is faulty': 1}
--> False:
Is Temperature sensor is faulty >= 1?
--> True:
Predict {'Material is not sufficiently heated because the heating cartridge is faulty': 1}
--> False:
Predict {'Material is not sufficiently heated because the heater is not switched on': 1}
--> False:
Is AK 4xx installation gun/gun is not closing properly >= 1?
--> True:
Predict {'AK 4xx installation gun/gun is not closing properly because the pintle seat is damage': 1, 'AK 4xx installation gun/gun is not closing properly because the valve bolt is damage': 1}
--> False:
Is Material contains hardened pieces (lumps) >= 1?
--> True:
Is Air in the system >= 1?
--> True:
Predict {'Material hose is blocked because the material contains lumps': 1}
--> False:
Is Nozzle is blocked >= 1?
--> True:
Predict {'Material unit is blocked because the material contains lumps': 1}
--> False:
Predict {'Nozzle is blocked because the material contains lumps': 1}
--> False:
Is Heater switched off >= 1?
--> True:
Predict {'No supply of material because the barrel is empty': 1}
--> False:
Is No supply of material from dosing unit/barrel pump >= 1?
--> True:
Is Barrel empy >= 1?
--> True:
Predict {'No supply of material because the stop valve is closed': 1}
--> False:
Is Stop valve closed >= 1?
--> True:
Predict {'No supply of material because the pressure release of barrel pump is activated': 1}
--> False:
Is Barrel pump compressed air supply off >= 1?
--> True:
Predict {'No supply of material because the barrel pump compressed air supply is not activated': 1}
--> False:
Predict {'No supply of material because the dosing unit is filling up': 1}
--> False:
Is AK 4xx installation gun/gun is not opening >= 1?
--> True:
Is Valve bolt is jammed >= 1?
--> True:
Predict {'AK 4xx installation gun/gun is not opening because the valve bolt is jammed': 1}
--> False:
Is Driver card in the controller is defective >= 1?
--> True:
Predict {'AK 4xx installation gun/gun is not opening because the driver card is defective': 1}
--> False:
Predict {'AK 4xx installation gun/gun is not opening because the SCA controller is switched off': 1}
--> False:
Is Material is escaping from material unit >= 1?
--> True:
Predict {'Material unit is leaky': 1}
--> False:
Predict {'Pneumatic drive is leaky': 1}for row in testing_data:
print ("Actual: %s. Predicted: %s \n" %
(row[-1], print_leaf(classify(row, my_tree))))Actual: AK 4xx installation gun/gun is not opening because the SCA controller is switched off. Predicted: {'AK 4xx installation gun/gun is not opening because the SCA controller is switched off': '100%'}
Actual: AK 4xx installation gun/gun is not opening because the driver card is defective. Predicted: {'AK 4xx installation gun/gun is not opening because the driver card is defective': '100%'}
Actual: AK 4xx installation gun/gun is not opening because the valve bolt is jammed. Predicted: {'AK 4xx installation gun/gun is not opening because the valve bolt is jammed': '100%'}
Actual: No supply of material because the dosing unit is filling up. Predicted: {'No supply of material because the dosing unit is filling up': '100%'}
Actual: No supply of material because the barrel pump compressed air supply is not activated. Predicted: {'No supply of material because the barrel pump compressed air supply is not activated': '100%'}
Actual: No supply of material because the pressure release of barrel pump is activated. Predicted: {'No supply of material because the pressure release of barrel pump is activated': '100%'}
Actual: No supply of material because the stop valve is closed. Predicted: {'No supply of material because the stop valve is closed': '100%'}
Actual: No supply of material because the barrel is empty. Predicted: {'No supply of material because the barrel is empty': '100%'}
Actual: Material is not sufficiently heated because the heater is not switched on. Predicted: {'Material is not sufficiently heated because the heater is not switched on': '100%'}
Actual: Material is not sufficiently heated because the heating cartridge is faulty. Predicted: {'Material is not sufficiently heated because the heating cartridge is faulty': '100%'}
Actual: Material is not sufficiently heated because the temperature sensor is faulty. Predicted: {'Material is not sufficiently heated because the temperature sensor is faulty': '100%'}
Actual: Insufficient supply of material because the dosing unit is leaky. Predicted: {'Insufficient supply of material because the dosing unit is leaky': '100%'}
Actual: Insufficient supply of material because the material hose is blocked. Predicted: {'Insufficient supply of material because the material hose is blocked': '100%'}
Actual: Insufficient supply of material because the material hose is leaky. Predicted: {'Insufficient supply of material because the material hose is leaky': '100%'}
Actual: Material unit is blocked because the material contains lumps. Predicted: {'Material unit is blocked because the material contains lumps': '100%'}
Actual: Nozzle is blocked because the material contains lumps. Predicted: {'Nozzle is blocked because the material contains lumps': '100%'}
Actual: Material hose is blocked because the material contains lumps. Predicted: {'Material hose is blocked because the material contains lumps': '100%'}
Actual: Material contains lumps because of air in the system. Predicted: {'Material contains lumps because of air in the system': '100%'}
Actual: AK 4xx installation gun/gun is not closing properly because of lumps on the pintle seat. Predicted: {'AK 4xx installation gun/gun is not closing properly because of lumps on the pintle seat': '100%'}
Actual: AK 4xx installation gun/gun is not closing properly because the pintle seat is damage. Predicted: {'AK 4xx installation gun/gun is not closing properly because the pintle seat is damage': '50%', 'AK 4xx installation gun/gun is not closing properly because the valve bolt is damage': '50%'}
Actual: AK 4xx installation gun/gun is not closing properly because the valve bolt is damage. Predicted: {'AK 4xx installation gun/gun is not closing properly because the pintle seat is damage': '50%', 'AK 4xx installation gun/gun is not closing properly because the valve bolt is damage': '50%'}
Actual: Material unit is leaky. Predicted: {'Material unit is leaky': '100%'}
Actual: Pneumatic drive is leaky. Predicted: {'Pneumatic drive is leaky': '100%'}In this way we achieve to create an app to diagnose the Failure Mode of an Sealer equipment. From here, further development can be made. For example, complement the training dataset with different data sources, create an interactive User Interface and add the capability to store new failure modes in the data set.
