The AI Ecosystem is a MESS

Why is it impossible to understand what AI companies really do?

Arize AI
Arize AI
Mar 17 · 5 min read
Image for post
Image for post
Traffic Chaos By sladkozaponi via Shutterstock

The State of AI in Businesses

Over the last several years, there’s been a rush to find out how to integrate AI into businesses, and it’s no secret that doing so could offer huge comparative advantages. But for all the hype, AI in businesses is still very much in the early phase.

Our team hails from Uber, Google, Facebook and Adobe where we’ve seen both the positives and challenges of deploying AI across business lines. Most companies don’t have the same resources to build in-house tools, deeply measure results and fund extensive research. Our goal with this blog is to use our in-depth knowledge of the AI space to make sense of the ecosystem, cut through the hype, and provide insights that can help you with your AI investment decisions across the pipeline.

In-House Data Science versus External Partner

Many companies are leveraging AI to upend areas of business by applying models to common business problems in various business verticals. We consider these companies as vertical business solutions — they sell software that is “powered” by AI. They target various teams such as customer service, marketing, finance and ops with a SAAS solution to solve their problems. The other set of companies are those selling software solutions to empower your Data Science, Data Engineering and ML engineering teams.

The rest of this blog is focused on companies in the latter space, AI/ML Infrastructure companies that are building software to help your teams build and deploy models themselves.

Complexity of the AI Space

One of the first problems we hear from enterprise teams is that it’s extremely hard to figure out the right problems to solve with AI. Even when they find the right problems, we commonly hear that most of the models that they develop never make it to production. It’s such a common problem that we know of a company building a model to predict whether other models will actually make it to production. You have to wonder if that model itself will make it into production.

At various Data Science team sizes and scaling points, you find that there are new unique complexities that arise out of the Data Science organization. Some of the complexities are really software engineering problems that have taken on a new form and others are unique to this space. All of this is compounded by the confusing suite of software tools and solutions that are built to “help” Data Science. Enter stage left, where you have a 100+ companies who are pitching confusing software solutions to “Empower your Data Science teams”.

How to Evaluate ML/AI Software Companies

AI/ML Infrastructure tools is an extremely crowded space. I can point to 100+ companies offering services in hyper-parameter tuning, deploying, MLOps, governance, control, auditing, regulation, explainability, performance tuning, you name it. Even as a technical person, I’ve been pitched by some ML tools companies, where after an hour pitch, I still don’t know what they do.

The nebulous quotes abound include “Achieve AI scale”, “Optimize AI”, “Empower AI”, “Take control of your AI” — but what do these software solutions actually do?

There is a simple mental model we use to section the space, place the solutions, and cut through the hype.

Image for post
Image for post
ML Workflow Stages Diagram by Author

Production or Pre-Production?

As you look at the ML/AI Infrastructure space, the first segmentation of products to make is pre-production or production.



The reason this segmentation matters is that as a Data Science team grows, different people in the organization are eventually responsible for production versus pre-production. Software solutions built for one function/organization don’t serve all people well across the organization.

Model Workflow Stages

The next segmentation of products to make is to where in the ML workflow does the product fit. Broadly, we see three main stages — Data Prep, Model Building, and Production.

Data Preparation involves preparing, labeling, and cleaning the data to be used for models. It’s where the lion’s share of time and energy goes in terms of using AI models, and it’s primarily under the domain of data engineers and data wranglers.

Model building involves feature selection, training pipelines, hyper-parameter tuning, explainability analysis, pre-production audits and evaluation. It is usually taken on by data scientists.

Finally, there’s production, which involves deploying models for inference (generating predictions), performance monitoring, troubleshooting and production based explainability. It’s usually in the machine learning engineer wheelhouse.

Each of these broad categories of the ML Workflow (Data Prep, Model Building, and Production) has a deeper set of features. Here’s a more detailed overview of the ML Workflow stages with product capabilities needed in each stage and companies offering features in those verticals today.

Image for post
Image for post
ML Infrastructure Platforms Diagram by Author

There is a strong debate whether each platform should be tailor made for going deep in the vertical or a horizontal solution makes sense. We will dive deeper into the various verticals and products in the space in upcoming blog posts.

Notes on AI Company Pitches

Up Next

We’ll be doing a deeper dive into the production segment of the ML Workflow. Arize AI is focused on making AI successful in production and we’re excited to share more tidbits about how to get the most value of your models.

Contact Us

If this blog post caught your attention and you’re eager to learn more, follow us on Twitter and Medium! If you’d like to hear more about what we’re doing at Arize AI, reach out to us at If you’re interested in joining a fun, rockstar engineering crew to help make models successful in production, reach out to us at!


  1. Blumberg Capital AI Toolchain
  2. Introducing Redpoint’s ML Workflow Landscape
  3. Oreilly AI Adoption being fueled by an improved tool ecosystem
  4. McKinsey Skill Shift Automation and Future of Workforce

Arize AI

Arize AI is the watcher, troubleshooter and the guardrail…

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch

Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore

Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store