Notes from Industry
Is MLOps Leaving the Software Engineer Behind?
MLOps delivers AI at scale for data scientists, but our silos still remain and that’s a shame.
Where We Are and How We Got Here
A few years ago, Venture beat reported that only 13% of data science projects make it into production. The primary reason? Companies hired data scientists but neglected to put proper supports in place to productionize their efforts. They siloed the data scientists from the rest of the organization, gave them some API keys, and asked them to weave gold. This failed terribly and, thus, in the mists of a deep learning revolution and new AI summer, the enterprise was left wondering how and why they couldn’t get a piece of the pie.
Introspection into why these data science projects weren’t yielding value lead directly to the evolution of MLOps. Through a combination of process and tooling, MLOps promises to make your data science teams efficient by enabling them to build, test, ship, and measure models faster. New roles, such as the Machine Learning Engineer, even evolved to help build and maintain MLOps infrastructure.
What MLOps Didn’t Fix
Don’t get me wrong, MLOps is a powerful technique to integrate AI, but it was a response to isolated data scientists that should have been embedded with product development teams in the first place. For all of the good MLOps has brought, it doesn’t (and wasn’t designed to) explicitly address this core issue of silos.
Silos are a systemic issue in many organizations, especially ones at scale or trying to scale, that often emerge from departmentalization and specialization. MLOps, unfortunately, can exacerbate this problem: once the data science team has their own process and specialized stack, it’s less likely they can integrate well with other product teams.
So while MLOps is speeding up their ability to ship new models, the silo is stifling their ability to work with other departments and find new AI problems to solve. They’re stuck iterating on the same problem they already have data and integrations for. In other words, the value add is the long tail: they’re chasing value by increasing model performance instead of finding new AI opportunities to impact the business.
An Example: The Software Engineer
The most obvious example of siloed AI expertise is the software engineer. Every software engineer I know is extremely interested in AI. Every one.
The enthusiasm is understandable: AI technology can add an element of efficiency, personalization, or magic to their applications that otherwise would be impossible to build. It can make their software better, and it’s fun to integrate it.
So why aren’t more software engineers practicing AI? Why have we left it to the data scientists and built MLOps to leave them to it?
This is particularly vexing when you consider a software engineer’s education. Look at any computer science bachelors curriculum and you’ll find AI courses right there along with data structures, algorithms, design patterns, and architecture. Then why have we let the software engineer role become embroiled in these core disciplines while effectively ignoring the AI component of their education? Why do we continue to ask interview questions about asymptotic complexity and not expected value?
It’s easy to excuse this by saying that software engineers aren’t practiced or specialized enough in machine learning, statistics, or mathematics principles to practice AI. But this isn’t true: as we’ve learned from the 13% problem above, it takes a lot more than the ability to derive backpropagation to ship AI. Just look at the “People of MLOps” chapter of Introduction to MLOps by Treveil et. al.: it takes a team of people to ship AI, a team that includes many different types of software engineers. In other words, getting AI into products requires skills beyond just building good models, and many of those skills are already honed by software engineers.
Enabling Engineers Can Be Your Ticket To AI
MLOps really shines with you have a large, crack team of data scientists itching to crank out model after model of deep learning powered money making goodness. But most companies I’ve talked to don’t, especially the startups.
Why? Hiring priorities. You have to get a product out and gather data before you can leverage it for adding value.
As a result, these companies are full of software engineers, great software engineers, wondering why they’re being left behind in the brave-new-world of the AI revolution. Concurrently, the leaders of these companies are worried about how to increase AI capabilities to remain competitive.
So here’s my hot take: leaders should enable software engineers to integrate AI into their products. Alleviate the engineer’s FOMO and grow your organization’s AI capabilities. Integrate data scientists (if you have them) into your product teams, and use MLOps to add efficiency instead of divide teams.
TL;DR
MLOps, while powerful, is currently designed solely for the data scientists. This is blatantly obvious if you look at any MLOps offering: they all focus on Juptyer notebook as the primary user interface. As a result, data scientists are siloed (once again) from the software engineers. This is a waste: software engineers are uniquely suited to get AI into products, and they’re eager to get involved. So please, I beg you, before you invest in MLOps, think about your organization and ask if it will speed up your people, or just isolate them.
If you like this story, please consider supporting me by buying me a coffee or signing up for medium using my referral.
