The Lean Machine Learning

We live in an amazing technologic era. In this era, technological companies (Google, Amazon, Apple, etc) rule the market and the world. When talking about technology I must mention the contribution of Machine Learning and Deep Learning. These 2 academic areas lead the tech world to innovate new startups and new products which solve our problems, improve our lives and make them more efficient.

Some startups in the field of Deep Learning (https://tinyurl.com/ms8abyds)

Innovating a startup is very hard and it might take a long time till it will be profitable. Although there are a lot of startups, the fact is that most of the startups fail. One of the main reasons for the failures of startups is that there aren’t enough customers who are willing to pay enough money so that the startup will be profitable.

So, in this long journey, the entrepreneur should first and foremost focus on one thing — prove that there is a customer who would like to use the product and be willing to pay for the product. The entrepreneur would like to prove that ASAP, otherwise, he will waste precious time and money. So what would an entrepreneur do when his startup requires research in the field of ML/DL, research that might be unclear, complicated, and cost a lot of time and money? In the next sections, I’ll provide the sweet spot between quick iterations and long research.

The Lean Startup

I can’t talk about startups without mentioning the amazing book ‘The Lean Startup’, by Eric Ries. In his book, Ries shows the basic concepts of startups:

MVP — Minimum Viable Product

MVP is the first product of the company. There are 3 main characteristics of MVP:

1. The creation of the MVP should be fast
2. The MVP should try to solve the problem that the entrepreneur defined, and give value to the customer
3. The MVP possibly won’t be efficient, scalable, or even profitable

The ‘Build-Measure-Learn’ Feedback Loop

• Build — the entrepreneur has an idea and starts by building it
• Measure — define business metrics that evaluate the success of what the entrepreneur has built, and measure them
• Learn — examine the results of the measure, then learn and draw conclusions of what to build next

Ries explains the importance of these concepts: “A minimum viable product (MVP) helps entrepreneurs start the process of learning as quickly as possible. It is not necessarily the smallest product imaginable, though; it is simply the fastest way to get through the Build-Measure-Learn feedback loop with the minimum amount of effort. Contrary to traditional product development, which usually involves a long, thoughtful incubation period and strives for product perfection, the goal of the MVP is to begin the process of learning, not ending it. Unlike a prototype or concept test, an MVP is designed not just to answer product design or technical questions. Its goal is to test fundamental business hypotheses.”

The Lean Startup by Eric Ries

Machine Learning in Early Startups

How does Machine Learning connect to all of this?

ML research can take a while and it doesn’t settle with the way startups work. I recommend an “MVP-like” research approach. Instead of designing a full research procedure, which is full of definitions, questions, models, data wrangling, and exploration, we should try to solve the basic research problem with minimal effort, time, and money. It is most likely that the ML model that we will probably build won’t be the best model with the best performance metrics, but it is just fine!

The main advantage of this approach is that we reach the BML loop extremely fast. After we have done the minimal research and deployed the model, there are 3 possible scenarios:

1. The performance is GREAT! — the customer is pretty satisfied with the product (and the performance of the model)
2. The performance is fine — the customer uses the product and is partially satisfied, though we get some feedback and extra work to do
3. The performance is poor — we didn’t solve the problem. In this case, too, we get some feedback and now we know what the customer prefers

Scenarios 2 and 3 are most probable to occur, though the most important thing is that we got feedback from the customer!

Cool, we are way in the BML loop.

Adaptation to Startup Way of Thinking

Ok so let’s talk business, how is the research gonna look like?

In the next figure we can see the classic `Machine Learning life-cycle:

https://www.javatpoint.com/machine-learning-life-cycle

This process that we can see in the figure might take a lot of time. Now I’m not saying that this is wrong, I’m only saying that this is too long when you don’t have a product yet.

My guideline, when we don’t have a product yet, is KiS — Keep it Simple. At the start of the startup, we would like to keep things simple, assume indulgent assumptions, just to get things going (and believe me, it’s really hard). Some key elements can delay us:

1. Complex problem definition — we tend to solve problems perfectly, define everything, know every detail. It might be true for the perfect product, which we are far from at the moment. Try not to overcomplicate things.
2. Complex data — the data isn’t perfect, and will always be, mainly after the first “data gathering” step. It might be noisy, inaccurate with a weird structure. It’s not great, but it’s not a disaster either. To improve the data we might have to work hard and it might not be worth it. Moreover, are we sure it’s gonna help? If so, how much, and is it really worth it?
3. Complex model — the world of Machine/Deep Learning is amazing and there are always better models in so many domains. Naturally, we will always want to use the SOTA (State of the Art) models. We should keep in mind that it might not be so easy to use. SOTA models can be heavy, slow, and learn for a lot of time. Moreover, to use it well we might need to read papers and documentation, which can take a while. Instead of using the SOTA models, we can easily use shallow models which are ready to use. The performance won’t be as good as the SOTA model, but it might be good enough to start.

Conclusions

All in all, each startup has its difficulties, data, and customers. In this article, I tried to provide “The Lean ML Startup” approach. This approach might not fit every startup, but I recommend that each entrepreneur should think about it. Remember, this approach is designed mainly for the first steps of a startup when it doesn’t have customers yet and it hasn’t built an MVP yet.

In my experience, using this approach, the research took 2 days using basic shallow models, the models were deployed on the third day and the entrepreneurs got feedback from the customer (who were pretty amazed) on the fourth day of work. The entrepreneurs knew that they succeeded in their first mission of proving that their product is worth something and has potential customers.