#3. This is why you are not iterating your ML project fast enough

Stop delaying that deploy, take risks to base decisions on production metrics

Nicolas Rodriguez Presta
Jun 10 · 4 min read

This is story #3 of the series Flight checks for any (big) machine learning project.

Ok, at this point i suppose you already have clear KPIs and the right team.

The next typical mistake in machine learning projects that we should avoid is taking longer than necessary to get something in production.

And having something in production and having a machine learning model in production are two different things.

Not all in a machine learning project its about machine learning
Not all in a machine learning project its about machine learning
Definitely not all is machine learning in a machine learning project

Let’s split this into 2 stages:

  1. To have something in production: this stage consists in solving the problem of integration with the system — at this point the model is only a mock, with some random output. This stage can be more or less complex and includes several things. If we are making a recommendation system, it includes integrating the frontend with the backend, ensuring that it scales, transforming the design of the KPIs into a dashboard, assembling all the necessary machinery so that AI ​​can make its appearance in an already assembled scenario.

For part 2, my advice is to try the simplest solution first. And when I say the simplest, I mean it. This includes:

  • Use few variables, few data sources and the easiest to integrate into production — having many variables and complex input data sources is usually a bad call.

Coming up quickly with a model helps in a number of ways:

  • To gain information on whether the whole pipeline — as well as its full integration — is running smoothly.
The real final profit is the integral of the profit curve at each point in time
The real final profit is the integral of the profit curve at each point in time

This means that once we have a first iteration, all the great improvement ideas are never going to be executed, right? Not at all!

It only means that we will be able to resort to more information in order to decide on the next best step. And iterate in that direction.

Sometimes less means more
Sometimes less means more

Without a quick baseline and grounded on assumptions and insights, we can’t do a marginal analysis of improvement and what’s worse, we run the risk of having costs and time skyrocketing in no time.

In short, the sooner we can base our decisions on concrete values ​​and less on constructs of hypothetical deductive reasoning, the better.

Mercado Libre Tech

Experiences and reflections from the Tech team

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store