Managing Quality Risk with Supplier Segmentation
by Anshul Bansal
One of the most versatile and interpretable machine learning techniques at a data scientist’s disposal is clustering. For the uninitiated, clustering is a statistical analysis designed to segment entities into distinct groups based on latent information in the data. Used across many domains for a variety of reasons, there are plenty of valuable use cases for clustering in the enterprise. In this blog, I’d like to talk about my recent experience applying clustering to a problem that every product company faces: supplier quality risk.
(This post won’t require advanced knowledge of clustering, but if you’re brand new to the concept or looking for a good non-technical explanation to share with others, you might enjoy my coworker’s blog post describing a fun application of clustering in golf.)
What Is Supplier Risk?
Today’s supply chains often involve an intricate, global network of vendors. While their size offers them many benefits, these logistics networks are often so efficient that they end up vulnerable to even minor disruptions. Supplier risk is the potential for loss faced by a company when one or more of its suppliers fail to deliver on their commitments.
There are several types of risk that could impact a company’s suppliers (e.g., legal, political, environmental), but the one we focused on in this study was quality risk. Quality risk is associated with product defects and service failures, which can hurt product sales and brand reputation.
How Can You Manage It?
With the size of modern supply chains, it’s hard to keep tabs on everybody. Much of risk management is trying to determine which suppliers to watch more closely than others, and a great way to make that determination is to use segmentation.
Often used in business contexts, segmentation is the process of partitioning a set of entities (in this case, suppliers) into distinct groups, with groups created such that each member tends to exhibit similar characteristics. Segmentation is widely used in industry to build Supplier Relationship Management (SRM) frameworks, which help firms strategically allocate their resources and tailor action plans for each of their supplier segments. For instance, a company might choose to spend more time and money on relationships deemed high-risk, whereas it may automate quality audits for low-risk suppliers. (This article provides some interesting details.)
Although there are various business rules-oriented approaches to segmentation, clustering is often a compelling alternative. It can process vast amounts of historical data, reduces the effort needed from subject matter experts, and is more versatile than standard business approaches.
Creating a Clustering-Based SRM Model
There are several steps to creating a clustering model for supplier risk management.
Problem Scoping
Many data science problems involve sifting through tremendous amounts of information, only some of which is relevant and high-quality. As a result, scoping the data effectively becomes a crucial part of an analytics project.
As any data scientist can attest, scoping can vary significantly from project to project, but three main tips come to mind:
- Figure out what your key stakeholders are interested in, and design a solution that is directly relevant to them.
- If you believe that whatever you’re trying to characterize with your segmentation model differs significantly when broken down in a certain way (e.g., geographically, product-wise, etc.), then make sure to split/subset appropriately beforehand for maximum effect.
- Data quality is rarely guaranteed, but do what you can to make it the best it can be.
Our problem’s data mostly concerned the results of periodic product quality tests. These included each product’s test score and any defects identified during the test itself. The dataset spanned ten years and several product categories.
With the above three tips in mind, we scoped our problem based on the following criteria:
- Business significance and product spend
- Products made by active suppliers
- Sufficient test count and frequency
- Recency of data
Feature Engineering
Imagine that you’re an artist who paints landscapes. Naturally, you have to travel into nature for your work, meaning that you’re limited in the number of paints you can take with you to capture your subject’s essence. In such a situation, it’s critical to select the right set— your art can only be vivid as the colors used to create it.
In clustering, colors are the features that fuel the model, and the resulting painting is the set of unique segments that the model generates. The features you create must characterize different aspects of what you want the clusters to capture.
We thought long and hard about the different ways our data could describe supplier performance, and came up with a list of features that could be classified into the following categories:
- Overall supplier performance
- Trend and stability in scores
- Tendency to repeat errors from previous low-scoring tests
- Performance relative to other suppliers in the product group
- Exogenous factors capable of affecting all suppliers within a product group
Initial Feature Selection
Brainstorming features is great, but having too many results in clusters that are hard to interpret and don’t generalize well to the larger population. Once you have a full feature list, you’ll likely have to winnow it down a bit (though don’t consider this feature set finalized — you’ll want to see what the data looks like after clustering).
Four good criteria to keep in mind when selecting features are as follows:
- Low correlation with other features (to maximize available unique information)
- Good heterogeneity (which means you can discriminate between observations based on said feature)
- High business relevance and actionability
- Good clusterability (more on that in the next section)
Exploratory Analysis and Model Selection
Once we had an interesting set of features, the next step was to perform some exploratory data analysis (EDA) to determine how well the data could be clustered. The first question of EDA in this context is to figure out the data’s optimal number of clusters. (While some clustering algorithms automatically determine the appropriate number of clusters, success is not always guaranteed and is dependent on the data’s distribution. In this post, we’ll focus on methods that require you to specify the number of clusters.)
The elbow method suggests picking the number of clusters by adding clusters until a new one decreases the marginal gain in percentage variance explained (a measure of compactness). By contrast, the silhouette method suggests picking the number of clusters that maximizes the average inter-cluster separation. In our case, both approaches indicated that the data’s cluster count falls in the range of four to six.
After choosing a number of clusters, you need to run a preliminary clustering model (use any technique you’re familiar with — k-means is an easy one if you’re unsure). Once a preliminary model is selected, cluster the data and visualize the results. We recommend using a dimension reduction technique (e.g., T-SNE, PCA) to visualize the clusters in two dimensions.
The plot below resulted from using the k-means algorithm with six clusters and then visualizing the results using T-SNE:
Next, visually inspect your plot to assess the effectiveness of the initial clustering. In our plot above, for example, each point represents a unique supplier, with its position in the 2D grid representing its similarity to other suppliers based on an amalgamation of the features fed to the model. The points are colored based on their preliminary cluster assignment, and it’s easy to see that there are distinct islands where like-colored points congregate together — evidence that our feature set can differentiate between suppliers based on risk behavior, just like we want. (There are some points that seem to have counterintuitive cluster assignments; however, this could just be due to the probabilistic nature of T-SNE.)
A Final Solution
When the segmentation solution is a means to an end (perhaps you want to build independent predictive models for each of your underlying segments), the design of a clustering solution(like the number of features or clusters) tends to be more statistics-driven. But in many cases, the segmentation is the end itself, in which case the model’s end-user interpretability and actionability becomes paramount, and its design becomes a more subjective process.
Consequently, the next phase involves multiple iterations of feature selection and algorithm selection to arrive at a valuable solution. The candidate models demand careful vetting from subject matter experts, and must incorporate end-user feedback to make the results as intuitive and actionable as possible.
When in doubt, consider the following two factors when evaluating a potential solution:
- Ensure the clusters are sufficiently differentiated, so action plans can be customized for maximum impact.
- Make sure each cluster is big enough to be meaningful. You want to capture behavior that’s widespread enough to be worth its own cluster, not hone in on a few misfits (which is a different kind of a study in itself).
After trying different types of algorithms (check out a list of scikit-learn’s clustering methods here), we converged on Partitioning Around Medoids (PAM). It’s very similar to k-means, but the centers of its clusters (the titular medoids) are actual data points, making it more robust to outliers and potentially easing interpretation for end-users.
The final solution from our study is summarized below:
We identified six clusters, which (after discussions with subject matter experts) we categorized into three broad risk levels:
- Low Risk
The “Leaders” were the best-performing suppliers, whereas “Troopers” differentiated themselves by excelling in a product group that was negatively affected on the whole by external conditions (e.g., made from raw material that was adversely impacted by external factors). These suppliers can serve as performance benchmarks, and require reduced human monitoring. - Moderate Risk
“Laggards” and “Average Performers” fell into this level. Managers need to address issues related to suppliers in this group on a case-by-case basis. - High Risk
“Negligent” suppliers were likely to have product defects recur from test to test, indicating a need for specialized training. “Volatile” suppliers performed poorly across the board; managers can choose to terminate contracts with them if corrective actions aren’t taken or are ineffective.
Finally, no solution is complete without good visualizations. A data scientist can build comprehensive, interactive tools for relationship managers that illuminate high-risk segments, track changes in risk behavior, provide high-level overviews, and allow users to explore underlying patterns.
Here’s a glimpse of the dashboards we built:
Conclusion
In this blog, we discussed how segmentation can be an effective way to manage quality risk, and how clustering can be used to create such segmentation schemes. We covered problem scoping, feature and model selection, and what constitutes a good clustering solution.
This is just one of the many ways that clustering can be used to make everyday business decisions more data-driven. If there are any other useful techniques that you use on similar projects, sound off below!
_________________________________________________________________
If you liked this blog post, check out more of our work, follow us on social media (Twitter, LinkedIn, and Facebook), or join us for our free monthly Academy webinars.
