Using Data Science to Predict Attrition: Retaining people using Artificial Intelligence
Identifying attrition way before attrition identifies you
What happens when someone leaves?
You as a leader are stressed out. You have to advertise, interview, screen and hire.
A new member joins after a month or two, but they do not know about what the job entails.
So more time is spent training and onboarding. It takes even more time for the new member to get acclimated to the new job and reach the same productivity as the previous employee.
Simultaneously you are managing the disengagement in the team. Why did their colleague leave? Is this going to happen to them as well?
What about you as a leader? How much time has this consumed from your work? How many dollars has it cost your company?
Cost of Attrition
A study by the Centre of American Progress found that the average cost to replace an employee is 20% of the annual salary of that employee.
According to a Deloitte study, in FY15, the highest voluntary attrition across sectors was seen in the IT services sector at 21.9%, whereas the lowest was in the energy and natural resources sector at 10.5%.
The high cost of attrition, combined with high attrition rates, makes attrition one of the most economically costliest problems for companies.
In the United States alone the cost of attrition is $536 billion per year.
What can we do?
Enter Artificial Intelligence
Artificial intelligence has been rapidly adopted as a revolutionary tool for tasks involving prediction and has profoundly transformed several industries.
Hence it is no surprise that several companies are trying to predict attrition using data science and artificial intelligence.
IBM’s Attrition analysis
According to Ginni Rometty IBM receives more than 8,000 resumes a day, but roughly 35,000 workers, know who in the workforce is currently searching for a new position. IBM artificial intelligence technology is now 95 percent accurate in predicting workers who are planning to leave their jobs.
As we can see in the above graph they got this accuracy using many data points like employee job satisfaction, age, job role and etc. AI has so far saved IBM nearly $300 million in retention costs, Ginni Rometty claimed.
So there’s our solution!
Collect a ton of data and analyze it! Easy right?
Not so fast…
Problems faced in Attrition analysis
The two most important factors to have an accurate prediction of attrition is to have a high volume of attrition data, which is also feature rich.
However, these are some challenges that most organizations face while collecting this information:
Lack of Volume
It is not always possible for all the companies to collect this huge amount of feature-rich data about their employee's insufficient historic data of attrition to predict future attrition.
Financial Constraints
Collecting data requires financial investment to hire quality data scientists and domain experts who understand the factors that involve attrition.
Time Constraints
Collecting data involves interviews, questionnaires, forms, etc. So individuals have to dedicate their time if they do not possess rich data or do not have automated tools to collect it.
This can be time-consuming, which increases the cycle time of predictions, and reduces the accuracy as a result.
Employee Privacy
Not all employees will consent to have their data collected and analyzed. This leads to inconsistent sets of data, and as a result, also impacts data volume.
How we are collecting data
Our simple solution to the data collection conundrum
The platform we have built at Up Your Game collects feature-rich performance data on an ongoing basis.
Performance data can give you a myriad of features such as employee sentiment & employee workload etc. that can predict not only attrition but also engagement and burnout.
We believe that performance data, combined with employee characteristics can give a robust model to predict attrition.
We’ve tackled the problems. Now what?
Analyzing attrition data
Analyzing data is the easier part.
AI Tools
Artificial intelligence has become so democratized that there are several tools today that help you build predictive models by barely writing any code. Here are some:
AI-powered HR Platforms
The other alternative is to utilize HR tools with this intelligence baked in.
The platform we have built at Up Your Game not only collects data from employee performance, but also has an artificial intelligence engine that is constantly analyzing that data, and making predictions whenever it achieves a high enough accuracy.
Once you have selected your weapon, all that has to be done is to specify your problem statement.
For this case that would mean:
Predict whether an employee is going to leave OR not
Thus your data would have to have a column that displays whether or not your employees have left:
Then we can use this variable to predict whether your current employees are likely to leave.
But we have a Data Science team
Choosing the right AI algorithms for the job
So based on our experience at Up Your Game we have tried and tested several models for attrition prediction using a number of predictors. We have found that certain models work better than others in certain cases:
Small amounts of data
Ensemble models like Random Forest and XGBoost work best for smaller but diverse amounts of data. While these give an accurate prediction they are harder to interpret i.e. harder to know what patterns are picked from the data.
Large diverse data
Using Deep learning models will more useful for large data sets.
Moreover, they are easily interpretable i.e. will also allow you to see how and why the prediction has been achieved.
The only constraint is that deep learning models need more data otherwise they cause problems like overfitting and they end up giving inaccurate results.
Conclusion
We can conclude that the most important thing in attrition prediction is collecting the relevant feature-rich data while utilizing robust tools to analyze it assuring an accurate prediction.
If you like this article then kindly give claps and if you have any questions please fill free to ask in comments.
Until my next blog
Happy Learning.
