How to sell MLOps in large Organizations?

Co-written with Raphaël Hoogvliets

We all want to be heard by our higher management, for our goals in ML projects. This week, we’ll give you tips on how to get buy-in for MLOps initiatives, in a non-tech company where your ML models are still considered nice-to-have, not need-to-have.

Disclaimer: These are the tips that worked for us, which might also work for you.

Are organizations investing in MLOps?

Well, we as ML Engineers writing for an MLOps blog might be preaching to the choir. We put forth our arguments and eagerly nod to each other in agreement. In the outside world, reality might be very different. We have found that it is hard to persuade decision-makers to invest in MLOps capacity and resources. MLOps is often not easily understood and perceived as a nice-to-have, and maybe (too) state-of-the-art toy to have. It depends on the context and scale of the organization and what the impact of good MLOps practices and tools will be. However, what is true for most organizations, regardless of scale, is that investment in MLOps will only be a relatively small part of the cake.

Consider the following. Once the C-suite has chosen to become “data driven” total investments in data capabilities are often huge. Resources might be needed or already put into:

• Hardware for data gathering
• People for data gathering
• Data ingestion pipelines
• Expensive external datasets
• Setting up infra and security
• Building a DWH
• Ensuring data governance and compliance
• Hiring expensive data scientists to develop models

And then there’s the organizational side of things, trying to change business practices or processes with ML solutions. People are called away from their daily routines to participate in these innovative projects for all kinds of meetings. Yes, there will be meetings and meetings on meetings: kick-off, check-in, sit-down, stand-up, brainstorm, breakout session, dragon’s den, shark tank, town hall, one-on-one, status update, retrospective, refinement, team building, training session, budget check, product review, all-hands.

Well, you get it.

This requires so much time, money, effort, energy from an organization. So much invested to get everything done aligned, working and implemented.

But then finally, the cherry on top, the actual MLOps. Mmm, organizations somehow seem a bit reluctant to invest in it. The one thing that will make sure the results from all these huge data efforts and investments will stay available, dependable, and tested. “No, this is sounding a bit too abstract. We don’t have the money and time for this now.”

Sound familiar? If so, we are going to try to help you to change that. How can you convince your organization to invest in MLOps?

1. Collect all the pain points

Before you present your MLOps concept, find the pain points and challenges within data science projects and teams. Talk to data scientists, product owners, and stakeholders in your organization. Gather all the complaints, issues, the pain points that can be solved by MLOps, such as:

Time to deployment: How long does it take to go from the PoC model, to the mature ML solution, to the actual solution running in production? For some organizations, this takes over a year. That’s a whole year your amazing model or “AI” is not adding value. We have seen solutions that add over >1m €. This is non-tech companies, not even digital native businesses. The Germans have a word that is the opposite of homesick: Fernweh. It’s that feeling you get when you want to be somewhere else, somewhere sunny and warm. We want to introduce a new term: business-weh.

Poor quality deployment: Organizations that are new in data science struggle to deploy ML models into production in a reliable way. Is the deployment automated? Is there version control? How about automated testing? Or are you even deploying notebooks? Check our minimum components article to compare your existing solution to how it should be: The mininum set of must-haves in MLOps.

Barely or non-existing model monitoring and maintenance: Poor deployment often leads to poor monitoring and maintenance. Using the right tools and processes is challenging without MLOps or ML team. As a result, you might have models giving wrong predictions for an extended period, or APIs failing without detection.

Lack of collaboration: Usually there is a gap between data science teams and IT operations. ITOps or DevOps has limited or no knowledge in ML products. This causes delays in deployment, a long time to solve bugs and issues in production. MLOps can fill this gap.

External parties: Outsourcing data science professionals for model development and deployment is a common practice within large organizations. While it offers many advantages, it also comes with drawbacks. External people often come and go, resulting in the loss of valuable knowledge. Solving issues or implementing changes costs more than having it solved by internal people. So, encourage your decision-makers to form an internal MLOps team. When done right, MLOps will result in templates, standard operating procedures, and better documentation. It will ensure the continuation of added value through data science.

2. Educate people

Be ready to teach people about what MLOps is and how it helps your overall business goals. Platform engineers, DevOps engineers, product owners, and managers will lack to understand. And let’s be honest, we cannot blame them. Grasping the full picture is a lot to comprehend. Talk to them in their accessible language. Try to think about it from their perspective. To an engineer, MLOps is a technical set of principles and tools, but for each stakeholder it has a different important impact: mitigation of risk, speeding processes up, increasing uptime, decreasing cognitive load, compliance, increased YTD revenue, etc.

Within our organizations we have organized workshops to explain what MLOps is. Our target audiences contained data scientists, product owners, team leads, platform engineers and DevOps engineers. From those workshops everyone took what mattered to them the most, but only because we handed it to them on a pretty platter.

3. Present before and after scenarios

You’ve collected all the pain points, and you have planted the seeds in people’s minds by showing what possible results it could bring. Now you are ready to build a compelling case for MLOps. You should clearly show how MLOps can solve these challenges and deliver tangible benefits to the organization. It is time to get concrete. Some examples from our own experience:

Less cost: Explain how MLOps can reduce operational costs by using the right tools for model development & deployment and automating tasks. Managers love to learn about cost-saving measures. We showed how wrongly the resources were used, due to the lack of knowledge in ML and how much more money was being spent. Give people numbers, even if they are ballpark figures. And when they are ballpark figures, add a disclaimer that they are exactly this. Your audience won’t care. People love a positive story translated to cold hard cash.

Fast deployment: Focus on how MLOps can speed up the deployment of ML models. With standardized, fully automated model deployment, you can deploy models quickly. Before we implemented MLOps, it took 6–8 months to deploy models, we reduced it to 1 month with our standard approach. Talk about the cognitive capacity (headspace!) and time this frees up for your co-workers. Throw some great ideas out there about what other useful things they could be doing with that time. This will be a triggering point, either people will love the idea, or come up with better ideas of what to do with their newfound freedom. Either way, they will start buying in.

Better collaboration: Show how MLOps improves the collaboration between data science and IT teams. If you are lucky you might already have teams or people dedicated to bridging this gap. If not, you just have a disconnect between data and IT, making your life hard in all kinds of ways. With all our experience implementing MLOps, building a better way of working with platform teams and DevOps engineers was an organic part of the process. We showed that we, as ML Engineers, are capable of creating resources, automated pipelines, and deploying applications in production. In this way, we were able to build trust and get more autonomy and much-needed rights to infra and tools. Eventually, as MLOps teams we became responsible for all the bells and whistles required for ML model deployment and maintenance. This is a big step up from the usual status quo where your urgent requests appear somewhere on the overcrowded backlog of a platform or cloud engineering team. Teams like this are usually servicing many departments that are all pressuring them with their “super urgent” requests. Moving away from this and being at the steering wheel drastically decreases throughput time and impediments. This won’t only save time but will increase the happiness of employees and parties involved.

Less risk: Mention how MLOps can help identify and address model drift, which ensures the models will continue to perform well. No unusual predictions will pass by unnoticed. We took an example use case where a model predicted a weird personalized offer for some time, and no one was able to find the root cause.

4. Prove it

Show an example use case, where MLOps can make a significant difference. You could explain it verbally, but it is not as strong as showing facts, or the actual benefit of choosing a specific approach. Back your proposal with tangible results from a PoC, or a similar use case from a competitor, or an organization in the same or related field. In this way you will build trust and earn credibility. We have successfully gained traction in the past by doing a PoC: we took one ML model that was poorly deployed and migrated to our proposed solution. It saved a ton of money and that was the awakening for many stakeholders.

5. Set up your team

Okay, you should now have resources freed up so start MLOpsing. Next is one of the defining steps of your organization’s MLOps transformation: finding the right people. You need to have capable people or teams that will build and maintain MLOps practices effectively. There is one rule we like to go by at this stage: quality over quantity. It is far better to have 1 senior and 1 medior engineer, or even 2 medior engineers, than a bunch of juniors. In data analysis or data science, you can kind of wing it in a lot of non-tech companies. Open a can of trainees and build something. For engineering this is not the case! Engineering requires experience, sound principles, and knowledge of a wide variety of topics. If your leadership is enthusiastic about solving the challenges with a traineeship or even intern program… stop them. It has a remote chance of working, but even if it does, it will likely create a ton of technical debt and won’t be secure. We started all our MLOps projects with small teams (two people) and migrated several projects to our new design first. Only later the teams grew and practices were fanned out to the rest of the organization.

6. Keep on keepin’ on

It doesn’t stop once you have gotten buy-in from all parties and have set up your initial little MLOps team. It is merely the beginning. Now it will be time to start a successful adoption from each existing solution to the next one. We kept our stakeholders happy and more teams wanted to collaborate with us. Eventually, we got more responsibility, and more people working with us for MLOps. And after that, a new tipping point might be reached where more capacity is needed and you need to shop around again while maintaining all your solutions to make sure they stay up and running, and are reliable. Once you have enough scale, you really want to add an “inside-to-outside person” as we described earlier in our article The perfect MLOps team, in this case that could be a good Engineering Manager.

It takes time and patience to sell MLOps in large organizations, and there might be limited resources available for MLOps in small organizations. Each comes with its own set of problems. Selling MLOps bottom-up is hard, but it can be done. In the end, it really depends on your organization’s governance structure and internal politics. We hope the pointers above will aid you in realizing your goals. Just remember: once you start unlocking the benefits of MLOps practices, you’ll get more trust and support. Stay persistent, be clear. And good luck.