Rapid Prototyping in Machine Learning
Adoption of Machine Learning is rapidly growing among small and big companies. The tools are available, people are excited about them. I imagine we are ready to conquer the world But are we?…
In this blog post, we will discuss the following,
- How to select the right use-case for a machine learning solution?
- why do we need Rapid-Prototyping in machine learning?
Choosing the right use-case
| Pandora Box of use-cases
Don’t waste your time on Challenges
Most times even though we have every resource available at our disposal, we might not have a use-case worth solving. i.e. how do you choose a use-case that really converts to some tangible benefits?
With Many use-cases comes extra complexity.
It's great to have many use-cases to build solutions for, but it doesn't translate well in reality. In fact, solve an easy/simple use-case first, they will provide results sooner, don't get attracted towards a complex beast, their time will come.
Rapid-Prototyping in machine learning
| Once your target is locked, what's your strategy?
Imagine spending a long time solving a use case and finally cracking it. In the time we have been building a solution, the priority has changed; the use-case isn't important anymore for the stakeholders, or worse something happened in the market you just didn't notice or you were too far into the project to make any significant changes. That’s terrible!
| Rapid-Prototyping as a solution
At the company where I work, we develop a deployable machine learning solution in 5 days. We call those 5 days as Spikes. A Data scientist involved in a spike is not allowed to engage in emails or meetings, what they are allowed to do is code, discuss solutions, brainstorm, eat pizza and sip chai/coffee…
“We tell our stakeholders, we are building a hut before we build a castle.”
note: Developing Machine Learning solutions from scratch differ greatly from Kaggle like competitions (more on it on another post, another day)
| 5 days of Spike
Day0
A Spike can have up to 10 people (5 SMEs + 5 Data Scientists) involved in solving 5 different use-cases. However, before we begin with any kind of modeling, almost a week worth of time is spent on collecting the data that could be useful to build a solution. The SME works together with the data scientist to gather the data. The SME plays a very important role as he is closer to the business and the data. i.e. he has better domain knowledge than others.
Day1
This is the day when we are sitting in the lab, away from all the distractions. We begin by looking at the objectives of the use-cases and then we create a quick one-pager to share with everyone. The document explains the following 7 items.
1| Translation of the business problem into a data problem.
2| The top 3 questions, the business would like us to answer.
3| The data collection: product data, customer data etc…
4| Any interesting new feature that we want to test.
5| The Algorithms that we plan to test. (Trees, Regression, SVM etc..)
6| Model interpretability is important too. (PDP, LIME, SHAP)
7 | Metrics that we will use to validate our models. (AUC, R2, F1-score)
While we explain the above points to everyone, most times people in the lab have great suggestions to give, thus this brings more collaboration and some great ideas. Following the interaction, we set up our systems, libraries and begin initial exploratory data analysis (EDA).
Day2
At the end of day1 we have a good idea about our data, so day 2 is spent on feature engineering and variables selection/dimensionality reduction. A lot of the feature engineering comes from the SME because of his extensive domain knowledge. This process helps in creating some important features that support in building a better model.
Day3
By the 3rd day, we have a working model, but it might have ~100 features, i.e because we started with a dataset that had ~500–900 features. Day3 is spent on further reducing the dimensionality and on model interpretability.
Day4
It's model graduation day because we summarise all our findings, model performance and its interpretability in a document on this day, we also explore a bit more to see if we can get any other potential insight from the data that stakeholders never asked for.
Day5
This is the final day of the Spike, where we go back to our stakeholders to explain them everything we have been building for the last 4 days. We get a lot of questions, which is a good sign! If the stakeholders are confident about the model, then we make plans to deploy and test it, with a plan to improve it further with new data.
We built the hut we promised, but a lot needs to be done…
Summary
The Rapid Prototyping helps in creating a minimum viable product(MVP) in a short time. If we and the users like it, we iterate and build a better one. Thus this way you keep improving without losing sight of the market and everyone has the right expectation.
I hope you like the blog, please comment if you have any constructive feedback.
