How to Close the Machine Learning Adoption Gap in Industry?

Jimmy E. Chan

Published in

Castodia

6 min readJan 4, 2019

An analysis based on a survey of 100+ AI companies data scientists and AI researchers

Introduction

Artificial Intelligence (AI) has progressed from being simple statistical models to powering self-driving cars, managing stock portfolios and predicting budgets. Companies are extracting value from machine learning (ML) by improving their decision making and enabling new capabilities. For example, a Canadian hospital is using ML to predict the amount of money they can recover from the government for individual surgeries, allowing them to start planning financial operations right away. Before using ML, the process took over three months, between figuring out the billing codes, sending them to the government and getting the approved amounts back. Now accountants know — with a 95% accuracy — which codes will be approved and how much money they will get back for the surgeries before they are even started.

So what’s the problem?

While there is still a lot of room for ML adoption to grow, there seems to be a market failure. On the one hand, there are a lot of advanced ML models created and published that could potentially solve business problems and enable new ways to perform tasks. On the other hand, it seems quite difficult to find and adapt those models to extract value for businesses. What’s really happening here?

1. There is a misalignment between market need and research focus

Researchers are constantly pushing the limits of what is possible with ML. However, the needs of the market often do not align with the interests of researchers. Scientists are mainly focused on state-of-the-art algorithms that could help secure publication and government grants, but may not necessarily solve an immediate business problem.

Furthermore, even if their research has a commercial potential and solves business problems, most scientists do not attempt to bring their innovations into the market. Researchers open their publications and source code to the public, but rarely optimize them for usability or reproducibility. In addition, academic publications are mainly focused on theory, rather than application, and the accompanying code is not production ready or scalable. In rare cases where researchers attempt to commercialize their work, they still experience challenges in building a strong business case.

Because of these reasons, many great innovations coming from AI labs either never make it to the market or their adoption is significantly delayed.

2. Companies lack the skills required to bridge the gap between business needs and ML capabilities

Although ML is a relatively accessible technology since much of the infrastructure and tools are open sourced, only a small portion of businesses can benefit from it today.

One of the main reasons for this is lack of expertise. Because this industry is relatively new, finding the right people is a challenge¹. There are not many experienced specialists available and most of them are already employed by major companies or research institutions.

Second, data scientists tend to specialize in different areas, but many businesses do not understand those differences. For instance, the roles of “data scientist” and “machine learning engineer” are often confused. Because of that, many companies make the mistake of hiring people with theoretical backgrounds to do programming, or programmers to build ML models. This mismatch often results in a poor outcome.

Third, to deploy a working ML solution, a company needs a team of at least two specialists: one with theoretical expertise to develop an ML solution for a given problem and one with programming expertise to productionalize it or integrate it into other products. In some cases, a third person is required to set up the right infrastructure and deploy the models. However, such team is expensive to support, especially for startups.

Finally, another obstacle is simply the lack of awareness about the capabilities and limitations of ML. Most businesses have a hard time mapping business problems to ML capabilities. Because of that, many businesses do not bother to seek for solutions that involve ML. Even when they have data scientists on their team, they either underutilize their potential or ask for solutions that cannot be practically solved with ML today.

3. Data scientists have difficulties keeping up with rapid advances in ML

Data scientists have their own set of challenges. Since AI is such a fast moving industry, the models developed today may be irrelevant by next year. Because of that, data scientists must always stay up to date with the latest advancements in the field. This poses a challenge since there is no easy way to test a novel algorithm other than by replicating it or re-implementing it, which is a computationally expensive and a time consuming process. Furthermore, ML researchers do not always have time to provide support for their work².

Often, researchers develop their algorithms using frameworks that are different from the ones used by the industry (for example, Torch vs. TensorFlow), so data scientists have to rewrite the code. On top of that, data scientists have to battle errors from version mismatches and environmental setups. Only then can they start testing whether or not this new algorithm could be used to solve their problem.

Another challenge for many data scientists is the nature of deep learning (DL). Over the past 5 years, most of the advancements in ML came from DL. However, data scientists find it more difficult to explain the key factors that were taken into consideration by the DL model. DL models create their own mapping of inputs to outputs³, in contrast to classical ML, where data scientists predefine these mappings. This lack of explainability makes it difficult to build a business case for the management team who then make decisions based on the model predictions. This problem is especially important in financial and health care sectors, where auditability is crucial given regulatory requirements such as the GDPR.

Because of these, only big and established tech companies can fully benefit from the progress made in the ML field today.

How do we close the gap in ML adoption in industry?

1. Developing a standard way to find, share and use models

To make ML research more accessible and accelerate its adoption, there needs to be a standard way for researchers to share their models and for developers and data scientists to easily access them without the need for re-training or complex environment setups. This could also be a way for researchers to track usage and further improve their models, as well as capture economic value from their innovations without much additional effort.

Researchers should be encouraged to share their work using the tools adopted by the market, such as standard frameworks within a container technology such as Docker. Furthermore, researchers should be aware of the type of challenges the market faces. A good example of a player making progress in this direction is Kaggle⁴.

2. Business managers must understand capabilities and limitations of ML

Businesses should invest into increasing their personnel’s understanding about the type of problems ML is good for, as well as what it can and cannot do. This can be achieved by investing in live training programs or online courses (MOOCS), attending ML conferences and interactive workshops that can help them recognize areas where an ML solution is ideal.

3. Investing in technology to improve data science workflows

To aid market adoption of ML technologies by developers, there needs to be an increase in investments in out-of-the-box solutions or tools that simplify the implementation and deployment of ML. This would enable developers even without a strong ML background to benefit from it. Facebook is an example of a company that has heavily invested in making ML accessible internally through their FBLearner platform⁵, which automates the building, training and scaling of ML algorithms. This platform is used by more than 25% of Facebook’s engineering team and has trained over a million models⁵

Acknowledgements

We would like to thank Creative Destruction Lab for their support and for connecting us with mentors and advisors.

Also, we would like to especially thank Amy ter Haar and Ozge Yeloglu for advising us with this project and Kunal Khosla for providing business and strategy support.

Furthermore, we would like to acknowledge the following individuals for their valuable feedback: Afiny Akdemir, Aidan Kehoe, Aisulu Saliyeva, Christopher Ing, Ehsan Amjadian, Nikita Varabei, Qasim Mohammad, Robert Aboukhalil, Royal Sequeira, Tomas Babej, and William Labonté.

Thanks to the 100+ companies, research groups, and individuals that participated in our survey and met with us to discuss the state of AI/ML adoption in industry.