New Research from the MIT-IBM Watson AI Lab Reveals How Work is Transforming
Rapid advancements in the field of artificial intelligence (AI) are uniquely poised to transform entire occupations and industries, changing the way work will be done in the future. The advent of AI will very likely shift the demand for labor skills. It is imperative to understand the extent and nature of the changes so that we can prepare today for the jobs of tomorrow.
New empirical work from the MIT-IBM Watson AI Lab uncovers how jobs will transform as AI and new technologies continue to scale across business and industries. We created a novel dataset using machine learning techniques on 170 million U.S. job postings. The dataset and research, The Future of Work: How New Technologies Are Transforming Tasks, allow us to extract key insights into how AI is shaping the future of work.
Insights into the Future of Work
This unique dataset allows us to study the changes in demand for tasks across occupations or wage groups. Further, we can tie the demand for tasks to their suitability for machine learning, deriving key insights into how AI is affecting the U.S. labor force. For example, we find that there is a small, but statistically significant negative correlation between the change in demand and suitability for machine learning. This may likely be due to the early adoption of AI and machine learning, indicating a fundamental shift in the way work gets done
We also find a shift in tasks from mid-wage occupations to low- and high-wage occupations, at a rate of 4-to-1. Low-wage workers gained the most in health care, public safety, and administrative tasks. High-wage workers gained the most in design, IT, and legal tasks.
Beyond these larger aggregations, our dataset allows us to drill down and extract extremely specific insights about certain occupations or tasks. For example, we find that the tasks that gained the most among low-wage personal care aides were care giving, meal preparation, and communication. By contrast, the tasks that lost the most among mid-wage supervisors of retail workers were store management, staff management, and sales.
What We Did:
Occupation and Task Data
All jobs are made up of a series of tasks, such as cash handling, taking messages, snow removal, etc. To observe the demand for tasks across all industries — and how that demand has shifted — we analyzed a dataset of 170 million online job postings from 2010 to 2017 in the U.S., provided by Burning Glass Technologies. In addition to the text of the postings, the dataset included a taxonomy of over 17,000 tasks which Burning Glass grouped into 572 task clusters and 28 cluster families. The changes in the occupations during that period are characterized via the evolution of the task-shares within each occupation.
Demand for Tasks and Calculating Task-Shares
To understand how the occupations are evolving, we dove deeper into how tasks within them are changing. From the job postings, we get the occurrence frequency of each task in a given occupation. Using the tasks count in postings for each occupation, a time-series dataset is generated. This measures the demand from employers for workers who can perform these tasks. In our analysis, we incorporated wages and employment shares data from the Bureau of Labor Statistics (BLS), who publish annual statistics of the average wages and number of employees in each of the 964 occupations. We normalize the task demand time-series data by the share of workers employed in that occupation to derive the unique task-shares dynamics data for each task-occupation pair. For example, administrative tasks made up 0.004% of all tasks in management occupations in January 2010.
To the best of the authors’ knowledge, this is a first-of-its-kind dataset that presents the task-shares at a monthly frequency (from 2010 to 2017) for each task-occupation pair and could serve as a benchmark resource for several downstream analyses and insights. Task-shares can be aggregated to visualize how demand for families of tasks has evolved within an occupation family, or across different occupation families. The task-shares data can also be arrayed across the high- mid- and low-wage terciles, to understand how the task-shares of different task cluster families have evolved across high-, mid-, and low-wage occupation families during the period of 2010–2017.
Suitability for Machine Learning Data
To analyze the impact that AI is having on the labor force, we used the Suitability for Machine Learning (SML) scores generated by Brynjolfsson, Mitchell, and Rock (2018). This dataset comes from a Crowdflower survey, which scored over 18,500 tasks, as defined by the Occupational Information Network (O*NET). It uses a rubric that assigns a score from 1 to 5 for 19 questions, which altogether determine the suitability of a task for machine learning.
Using this dataset gives us an SML score for each O*NET task. However, the above task share data is calculated using the Burning Glass taxonomy. In order to obtain an SML score for each task within our set of job postings, we trained a word2vec model on the text of the postings, using DLaaS, a deep learning service within Watson Studio. We then projected the O*NET task taxonomy into this word vector space to identify the closest Burning Glass tasks for each O*NET task.
Summary
In addition to the insights already extracted, this study and dataset lays down the scope and foundation for detailed exploration of the evolution of occupations (and the tasks within) across different industries in the US labor market. The task-shares time-series data creates an opportunity to learn the dynamics of task and occupations, and, then quantitatively predict the task-shares for near future with confidence bounds.
Today, we know the change AI and new technologies will bring to the labor market is small, but real. To prepare for continued adoption and advancements in the technologies workers can reskill, employers can help employees retrain, and new graduates can learn the skills required to be able to execute the tasks of the future.
The Future of Work: How New Technologies Are Transforming Tasks includes work by Martin Fleming, Wyatt Clarke, Subhro Das, Phai Phongthiengtham and Prabhat Reddy. To learn more about our research read our press release or full technical paper.
