Machine Learning & Big Data in HR: Are they Overhyped?

Over the last decade, there has been many talks about big data and machine learning in HR. Are they overhyped? Oh, we are just looking at the tip of the iceberg.

In the HR world, things used to be simple.

In earlier days, HR’s primary duties were record maintenance and payroll. Over time other duties such as employee training, uniformity and well-being was added to their tasks. Later on, recruitment and skilled workforce selection was added to their duties.

Today Human Resource Management (HRM) is assuming a more critical role than ever before. These days, not only are they responsible for all the mentioned tasks, but they are also responsible for employee motivations, well-being and workforce development as well. The HR groups have more data in their finger-tips than ever before. They are actively attempting to align individual goals with the corporate goals and objectives. Furthermore, strategic HRM focuses on actions that differentiate the organization from its competitors and aims to make long term impact on the success of organization.

Data and decision-making based on data, is becoming more crucial. These days, HR organization not only have the traditional roles, but they also expertise in fields such as data science, data visualization and machine learning. So yes, the HR data age is just beginning.

Earlier last month, I attended SIOP LEC.

Earlier last month, I attended one of my favorite conferences, SIOP Leading Edge Consortium (LEC). I like this conference not just because the food tends to be great, but also the topics are engaging. This year’s theme was: “Talent Analytics: Data Science to Drive People Decisions and Business Impact.” Topics ranged from employee engagement to effective data visualization to best practices for candidate selection.

In one way or another, different talks discussed different applications of big data and machine learning.

This is one of the few conferences that shows we are just looking at the tip of the iceberg. And there is so much more to be done. The breakthroughs in computing have enabled a whole new class of analytics. New ensemble learning techniques have overtaken classical linear approaches to model building for most problems and data competitions. New algorithms such as deep learning have set new records in image, audio, and text classification.

The expectation for computers now is to outperform human experts on many different tasks.

The expectation for computers now is to outperform human experts on many different tasks including but not limited to: new modeling techniques for unstructured datasets, resume analytics, interview analysis and automated assessments.

I enjoyed all the talks at SIOP LEC. For me, three talks by Ben Taylor at HireVue, Allen Kamin at GE and Alexis Fink at Intel stood out.

Ben Taylor, Chief Data Scientist at

HireVue gave a talk: “

The super-human era: The latest in HR data science innovation.”

In his Talk, Dr. Taylor explained how our hiring models have evolved over time. “It started from simple to scary.” Dr. Taylor explained. First, we had the linear models, then we moved to the tree models, next we are using gradient boosting models, and finally, we are at the deep-learning models.

In 1800s, the linear models were the simple way of recruiting candidates. If you had good GPA scores, you were likely to land a job — Nice and simple. In late 1980s and early 1990s, the model evolved where hiring process involved other external factors such as the school you attended, the SAT scores and GPA scores you got.

In early 2000s, concepts of machine learning were coming out. Slowly machine learning techniques such as Gradient Boosting were created. Gradient Boosting is a technique where various regression and classification methodologies are used. The point is to generate a predictive model. An example of such model would be a combination of work experience and some personality assessment. Such model might predict that by combining these two, you would be able to predict if an employee is effective at their work.

We are at the verge of Deep Learning. Deep Learning or Hierarchical Learning is a branch of machine learning based on a set of algorithms that attempt to model high level abstractions in data by using a deep graph with multiple processing layers, composed of multiple linear and non-linear transformations. Such devices might combine one’s personality scores with one’s social media presence to predict certain outcomes.

Another great talk was by Allen Kamin, Manager of Organizational & HR Analysis at GE.

The talk was called: “Enhancing Data Availability to Improve Employee Experience.” GE Analytics group is on a path to understand their employees to great depth — anything from the usual demographic, compensation and performance data to gathering data on their employees’ social interaction.

Just like how Amazon uses various techniques in providing product recommendation, GE provides “Buddy program” where like-minded employees are recommended to connect. By using such program, GE is creating a more intimate culture and encouraging networking within their organization. Just like how edX recommends popular learning modules, GE’s BrilliantYOU suggests learning modules for their employees. And just like how LinkedIn provides job recommendation, GE “Career Explorer” offers job recommendation to their own employees. Among many things, this program reduces the employee turnover.

Another great talk was by Alexis A. Fink, GM of Talent Analytics at Intel.

As briefly described earlier, the HR analytics have more data than ever before. But what can you do with an ocean of data? In Fink’s talk: “From Optimism to Impact,” she provided the framework for her 7 key steps for data success. The process starts by asking the right question and identifying the right method. Next, we have to generate and analyze the data. Then, insights and action-plans have to be developed. And lastly, measure the result to ensure the action was effective. Dr. Fink explained that an ocean of data is not beneficial if we cannot obtain 1–2 useful insight out of it.

So, for all the folks who believe “machines will take over the world,” humans are the intuition behind the machines. Machines can crunch numbers. But humans have to find the insights.

So, is machine learning (ML) and big data a big hype in HR? I think we are just looking at the tip of the iceberg.

About SparcIt: SparcIt is a technology company with a focus on large-scale semantics analysis. Our flagship product is our well-researched and well-known assessments is SparcIt’s Creative Thinking assessment. Unlike traditional assessment, SparcIt’s unique feature is the use of open-ended exercises and automated scoring. Using a Watson-like engine, SparcIt’s patent-pending engine, accurately and efficiently grades the participants’ responses and provide a detailed report to the participants and the test administrators. Hence, it eliminates the major factors for not using such assessments.

SparcIt’s Creative Thinking assessment is fun, fast, automated, affordable and scalable.