Human Aspects of Machine Learning
As machine learning (ML) is being adopted more widely, human lives are being irreversibly transformed . Some ML uses are visible but many others are not even noticeable, already working behind the scenes.
First, our daily lives are becoming more influenced by “machine-generated” insights. The web pages you browse seem to know the next questions you would like to ask. Restaurant recommendations are becoming sharper and routes to your destination are optimized based on up-to-date traffic and road conditions, even more optimized than drivers can intuit based on their experiences.
Second, leading organizations now consider machine learning a requirement for success, the new normal for competitive advantage. This means that basic automation and optimization alone can’t help a business outpace its competitors. As a business leader, you need to continuously optimize your decisions based on how your customers and partners are responding and then improve how the system and the humans together can provide better offers.
Third, the expectation that data scientists will produce visible contributions to the business are higher than ever before. How can a data science leader move quickly from exploration to deployment while increasing the productivity of his staff and rest of teams?
Operationalizing ML to Influence Behavior
Your behaviors and purchase patterns are evolving based what you see or read on the web. Businesses are optimizing the offers based on each individual’s propensity for next actions. Gone are the days that segmentation-based marketing could give a company the edge. Industry leaders are now continuously adapting cloud-native applications for each individual user as ML algorithms can be used to drive continuous changes to applications. For example, when using a ride-share application, I might get one user interface that provides various the ride options, while my friend may simultaneously get premium options at different price points. By the same token, in some shopping applications, I’m likely to get offers for expensive shoes versus my friend.
Optimized offers with constraints based on physical and virtual behavioral patterns can only emerge when ML pipelines are deployed to adapt the applications and inform the business — wherever data resides, on premise or in the cloud with hybrid access to anywhere.
Consider two organizations that successfully deployed ML in production:
- Company 1: Dozens of models in production. Batch — updated every week
- Company 2: Hundreds of models in production. Real-time — continuously optimized
Company 1 can provide seasonal and weekly offers based on historical records combined with geographical and demographic patterns. On the other hand, Company 2 is more empowered to influence individualized actions through a set of application user interfaces based on the dynamic locational data, real-time context, weather data, social and news feeds, and other insights.
Because of the pervasive convenience and plethora of options, an individual will likely buy things and go places that suit their preferences and moods for the day, not their preferences and moods from a week before or even from last night. The capability to operationalize ML into continuous business activities is becoming one of the core competencies for organizations to stay competitive.
Align People with Domain knowledge, Data and Development
Data-savvy business leaders and analysts have an increasing appetite to know the state of the business. Key stakeholders need to understand patterns, correlations, and causality to the current trends by using ML and other AI techniques. As a result, we hear from our clients about renewed focus on the following:
- High interactivity in data visualization between business users and data scientists
- Out-of-box and other visualization capabilities extended from open source technologies to suit iterative analytic processes between developers and analysts
- Visualization as part of the end-to-end integrated data science lifecycle, not a disparate task
Imagine a scenario where business analysts are working with data scientists and developers on the correlations between weathers and air quality. Figure 1 provides high-level correlations across wind speed, pollutants, and varying degrees from good to severely polluted air quality. You can also see the level of air pollution (PM 2.5) for a foggy period in Figure 2.
If your developers are using Jupyter notebooks to plot results, you can also visualize from the data sources directly. The benefits of this integrated approach include:
- Increased productivity across the team
- Accelerated time to insight augmented with ML
- Exploiting skill sets across the business expertise, data science, data engineering, and application development
- Ease and speed of operationalizing with real-time user warnings that you can enable via applications and alerts.
Data Scientists at the Heart of Business
Skill shortage, turnovers, and high wages are known factors in managing data scientists. There’s a long list of desired data scientist skills including R and Python programming, data management, advanced analytics, business domain, development and proficiency in a plethora of third-party analytic solutions. It’s neither practical nor necessary for a manager to find each individual with all of the skills. Even if you hire that person, you can‘t expect him or her to stick around.
And of course, there’s no guarantee of a better business outcome since there are so many other complex factors involved. Instead, if you are leading the data scientist team, you not only want to keep the existing data scientists productive, but also need to empower and retrain other analysts and experts to contribute to the data science process.
Furthermore, a data science leader wants to bring all the capabilities together to feed and improve the pipeline of ML algorithms on a right set of data. For this reason, a data science platform approach to help data scientists and business analysts collaborate and share findings and conclusions becomes more important than ever.
Such platforms also need to be open and flexible so that developers can have access to ML as a service and integrate it into an application environment through a REST API. For instance, some of our clients use real-time scoring via individual API calls or via an integration with IBM streams. In this environment, the leader can also exploit 30+ out-of-box ML algorithms in addition to programmatic approach to building and refining neural networks and other techniques.
Next Steps
To learn more about the latest projects, ideas and research from our teams, follow us on Inside Machine Learning.
