The 3 Kinds of Context: Machine Learning and the Art of the Frame

“What do you do for a living?” That question used to have a pretty clear answer: “I’m a data scientist.” But lately, it’s gotten more complicated…

The three of us — Jorge Castañón, Óscar D. Lara Yejas, and Adarsh Pannu — do data science at IBM’s Machine Learning Hub, where clients from around the world bring us their mission-critical goals for turning data into knowledge. The clients range across industries from insurance to retail to energy to finance. On Monday, a manufacturer needs to use data from the quality control process to fast-forward the testing of components — with no forfeit of excellence. On Wednesday, a healthcare provider needs to isolate the real factors that tip patients from low risk to high risk. On Friday, a credit union needs to improve its loyalty offerings for retirees.

Most client visits last just two days. In those 48 hours, our job is to go from zero to insight. We set priorities, access and clean the data, fire up Jupyter notebooks, set up a collaboration environment with Data Science Experience (yes, this is a plug; it’s fantastic.), choose algorithms, build models, run and tweak the models, and generate visualizations and recommendations.

But none of that is the complicated part…

The 3 Kinds of Context

The complicated part is about context. We’ve learned that hopping from project to project doesn’t just mean hopping from one context to another — it means hopping across multiple kinds of context:

• Industry context
• Data context
• Transfer context

The first two are fairly intuitive. The third is less so. Let’s take each of them in turn.

Industry Context

Well before we dive into data and models, we ask clients to convey their domain expertise. These are people with a seemingly limitless understanding of the industry issues at play — and of the dynamics that are shaping the demands of those they serve. The more we listen to these clients, the clearer it becomes that each industry (aka sector, aka vertical) represents a problem space unto itself, each with its own goals for data: Healthcare clients tend to want to solve classification problems. Finance and energy clients tend to want to solve certain kinds of prediction problems. And manufacturing, transportation, and insurance clients tend to want to solve optimization problems. Are those tendencies cut-and-dry? Absolutely not. But combined with careful listening, they give us a place to start.

But then there are the limitations. Sometimes the limitation is about the client’s familiarity with machine learning itself. Healthcare and retail have been deep into machine learning for years, while other industries are just ramping up. (Interestingly, sometimes the less familiar the better, since some clients turn out to be sitting on troves of accumulated data — typically proprietary data behind the firewall that’s just waiting to be mined.)

And sometimes the limitation is about the need for interpretability. The algorithms and models we choose vary from client to client based on whether the models need to “show their work”. Our healthcare client needed more than a numerical prediction of risk migration for a given patient; they needed to know the factors at play and the weight for each factor. By the same token, banks, insurers, and government bureaus need to be able to assure watchdogs and regulators that their ML-driven automations are bias-free. To preserve interpretability for those industries, we might try to favor methods like logistic regression and decision trees. Where interpretability is less important — for example, in retail — we can jump into deep learning and other black-box approaches.

It’s only after we have our heads around that industry context that we start to puzzle through the actual data.

Data Context

After cleaning and formatting the data we get from clients, we’re looking for what kinds of ML models the data is capable of driving. And let’s be frank: some clients approach us with real problems that just can’t be addressed with machine learning and the data at hand, so first we talk through what’s possible. Once we have something tractable in mind, we can start to ask more questions: What are the inputs and outputs? What’s the plan for feature extraction? Should we use supervised or unsupervised learning? (So far, it hasn’t made sense to use reinforcement learning, but maybe someday soon.) Is the response variable continuous or a class that you want to predict? If you need a classification model, which variables help to represent the classes we’ll use? And so on. That work gets us a list of potential models.

But on top of all that, we also want some context about how data comes to the system in the real world. How much data? How often? As a stream or in batches? Not to mention questions about provenance, governance, and security. We seldom have enough time to go as deep as we want to with clients, but without some of that context, we might end up creating models that can’t actually be deployed, accessed, or retrained.

So, the industry information and the data take us a long way toward framing our efforts, but there’s one more angle we didn’t anticipate.

Transfer Context

The more time we spend at the Machine Learning Hub, the more we’re struck by what we’re learning about learning. Naturally, we want to come fresh to every encounter — but we still want to benefit from all the work we’ve done before. As we think about those trade-offs, we’re realizing that our daily work as flesh-and-blood data scientists maps onto a key aspect of the search for artificial general intelligence (AGI): transfer learning.

As the name suggests, transfer learning means trying to improve performance on a task by leveraging knowledge acquired from some related task. That’s something we do every day. How well we do over time will depend on how successfully we can discern the knowledge that we should — and shouldn’t — transfer from one engagement to another.

In that sense, the third context is really about our roles as data scientists and being aware of that context means being aware of our opportunities for improving our methods across a wide range of problem spaces — while also thinking of ourselves as learning machines that are prone to cognitive biases. Who knows, maybe the processes we develop at the Machine Learning Hub will offer clues to achieving AGI.

Art of the Frame

As much as thinking about these three contexts has helped us, it’s also reinforced the fact that machine learning is often more art than science. For us, it’s an art of emphasis and de-emphasis. It’s the art of finding frames to put around the world — whether that’s a frame around an industry, a frame around the data, or a frame around our own learning.

Whatever the frame, our hope is to enlarge and energize the features that matter — and to see them with fresh eyes.

For more about our work at the Machine Learning Hub or to schedule a session, reach out to us. We’d love to continue the conversation.