The Difference Between Machine Learning & SPC (and Why it Matters)
Written by Nathan Lai & Jean-Christophe Petkovich
One of the biggest challenges that comes with working on an industry’s cutting edge is that you’re constantly explaining what your company does, how that differs from the traditional approach, and why it’s better.
Granted, that sounds like a humble brag (“The hardest thing about being so amazing is constantly having to explain why we’re so amazing!”), but it’s something that every company with a disruptive technology or business model seems to go through.
Henry Ford’s famous quote about his customers wanting faster horses conjures up scenes of the frustrated industrialist trying to convince early adopters that a dumb machine requiring careful maintenance is superior to a (relatively) intelligent and low-maintenance animal with a proven track record stretching all the way back to the beginning of history.
To take a more recent example, Dr. John Olsen, who developed waterjet cutting technology, once commented that the most common point of resistance in the early days of his company was the computing power necessary to control the jet. Pitching to manufacturers in the early ’70s, Olsen was told repeatedly that, “Nobody will ever accept a PC on the factory floor.”
At Acerta, we face similar obstacles when it comes to machine learning, typically in response to these four graphs:
These graphs show the signals from an Acerta client’s end-of-line (EOL) transmission tests. (For more information, read the complete case study.)
The red line indicates a defective unit, which passed the existing EOL test but was still caught by our model. It’s a great visualization of the precision and subtlety of machine learning, but the question that often immediately comes up is, “Can’t you get the same results with SPC?”
Statistical Process Control vs Machine Learning
The subtitle for this section is somewhat misleading, because machine learning and statistical process control (SPC) are not mutually exclusive in the way that, say, chamfers and fillets are. In fact, machine learning has been proposed as a method for augmenting the construction and interpretation of SPC charts. However, because machine learning is the new kid on the block while SPC is a much more established tool, suggestions that the former can enhance manufacturing efficiency in ways the latter cannot tend to be met with skepticism. But we would argue that machine learning enables manufacturers to fill the gaps where SPC loses its efficiency.
Let’s start with when or why you might employ SPC. If you have a relatively capable process and there are key product/process characteristics that you want to control then it makes sense to use SPC for that feature. You would establish sound reasoning behind the sampling quantity and frequency based on your capability study or other process study to make sure you capture any statistical variation in the process.This allows your operators to react to issues if their X bar and R chart begins to show evidence of statistical deviation.
Machine learning is not looking to replace SPC, because SPC is a cost-effective process control technique, and it’s especially useful for machining operations. But the fact that every pump, torque converter, transmission, etc. gets an EOL test should indicate that SPC alone is not enough.
Let’s face it, even if you have reams of SPC charts showing good statistical control and in-process data that shows good capability, you’re still doing end of line tests. And no one can afford to 100% gauge every critical clearance feature (even when those stack-up features are likely on your SPC list). Machine learning won’t replace the SPC checks, but it can help you answer questions like, “Why do I fail my EOL if all my parts are technically in specification?”
One might wonder why we can’t just use SPC on EOL tests (i.e., apply statistical tools to aggregate data) to get the same insights as machine learning. Going back to the example above, the reason Acerta was able to identify that signal as originating from a faulty unit is because our method looks at the relationships between signals in tandem. We aren’t looking at the trends of one signal to see if it’s going out of statistical control (i.e. trending towards the control limit), we’re leveraging ML to find connections between the time-series of the signals themselves and, more importantly, to each other. A single transmission can outputs over 500k time-series datapoints in an EOL test. The value of machine learning is in finding the relationships between all of them and comparing those to all the other transmissions to find abnormalities.
This is simply not an SPC application.
Even if you were to employ SPC techniques on some of the more important signals (which necessitates identifying which signals are important), you would only know if you were trending in a certain direction around your control limits. But that does not necessarily mean you’re dealing with a defective unit. Looking at all of the connections between signals rather than focusing on the direction a single signal is trending over time is what distinguishes machine learning from SPC, respectively. It’s also what enables machine learning to yield actionable insights, whereas SPC can only tell you that your process is trending in a certain direction but not what to do about it.
From an automotive industry perspective, you only want to use SPC when you absolutely have to, e.g., you have a specific feature on a part that affects assembly downstream, but you do not feel the need to gauge this feature 100% of the time based on a capability study. You know that your process is relatively capable on this feature (between 1.0 to 2.0 Cpk) so you employ SPC and have a sampling frequency to keep the feature within statistical control to avoid assembly issues downstream. With machine learning, you have a complicated model that understands your process that helps provide you insight into root cause and further protect your product from the process, in addition to SPC.
Machine learning and SPC simply aren’t in competition, so why does it seem like they are?
Category Mistakes in Manufacturing
Imagine someone asked you to show them manufacturing as it’s happening, so you book them on a tour of the local automotive plant. Wandering through the enormous facility, they’d experience an industrial rhythm that’s been playing for over a century: whirring conveyors, clanging press brakes, the pop and sizzle of welds, a cacophony of countless pneumatic and power tools shouting over one another.
It’s a humbling experience to see the outcomes of millennia of technological progress all working together in highly organized fashion, but suppose your friend returned from the tour impressed yet strangely dissatisfied.
“That was all very interesting,” they say, politely, “But I never saw the manufacturing! Where was that machine?”
Your curious but confused friend has committed what’s known as a ‘category mistake’, which is exactly what it sounds like: erroneously characterizing something that belongs in one category as though it belonged to another. In expressing a desire to “see” manufacturing, your friend assumed that manufacturing is a process like fabricating or machining and hence that there would be a “manufacturing machine” somewhere on the factory floor.
There’s a similar (though obviously much subtler) mistake being made in the assumption that machine learning and SPC are competing approaches to quality control. A better way to understand the relationship is that machine learning is simply the next stage for SPC, the logical next step in response to the growing volume of data generated from manufacturing.
Statistical Process Control + Machine Learning
If machine learning represents the next step for SPC, then the natural question to ask is, “How do I know if I need to take that step?”
(Great Question! Contact Us so we can help you answer it.)
The simplest answer to this question is that if all the in-process checks and the way your process control plan is dictated tell you that your parts should be good, but you’re still running into assembly problems downstream, it might be time to supplement your SPC with machine learning. To put it another way, if you’re dealing with univariate data in a normal distribution, SPC might well be sufficient for your needs. On the other hand, if you’re dealing with multivariate data that does not conform to a normal distribution, you might need machine learning.
Generally speaking, the prerequisite for machine learning is having a large data set, and these days most manufacturers do. You could try to apply traditional statistics in such cases, but you also need to remember that the data was generated without any kind of statistical experimental design. So, when you draw your conclusions, you’ll need to account for the fact that your data is imbalanced. The high resolution that comes with big data means that any small difference might seem significant, even when it’s not.
In addition to the theoretical issues, there’s also the actual practice of SPC to consider. As with any task involving a human element, SPC runs the risk of being poorly executed. In many cases, it’s quite literally an operator with a calculator and some graph paper, sampling parts, calculating the average and looking for trends. Compare that to machine learning, where data that’s being generated by the production line already is leveraged for additional value.
Rather than viewing machine learning as a challenger to SPC, think of it as covering you where SPC doesn’t. As a final analogy, consider someone weighing the benefits of upgrading to a smart home. “Why do I need something that will turn the stove off automatically after I go to bed?” they might ask. “I already have a smoke alarm that will tell me if I forget!”
