5 Lessons Learnt from Starting Machine Learning Projects in the Enterprise

I’ve had the good fortune of starting a few enterprise machine learning projects since I picked up machine learning two, three years ago. -Some had use cases which were naturally compelling to their respective business or product owners, while I had to take a bit more time on others to flesh out their utility. Having documented my learnings, I would like to share 5 things I’ve learnt from getting such projects running.

1. Understand the Business and Help the Business Understand the Solution

Pre-trained NLP language models are pretty cool, and Google/Microsoft/Amazon and just about any other prominent tech company is on it. But until you explain what specific problems the technology can solve, users don’t really care. It can be a paradox for data scientists/machine learning engineers — much as we love being immersed in the latest technological advances, we should avoid falling in love with the tech at the same time. Instead, we should aim to understand the pain points faced by the business and help relevant users understand what machine learning solutions can do for them.

For example, I was working on a risk forecasting project where we surfaced the need to explain its underlying methodology only after doing the proof-of-concept. As it turns out, our business users were more concerned about being able to explain how the results were predicted. The reason they gave was that the need to know how the risk was forecasted was vital given the high stakes involved. , This is especially true with stock market forecasting projects, where some say the signals are a random walk. This led to us re-looking how the models were chosen, and we highlighted the models that had clear, easily understood underlying methodologies, with linear/logistic regression being a good baseline model.

It could even be a case where what the business user needs something that does not require a bespoke application. If all you need is a beautiful graphical representation of your data, something like Tableau would very well serve your needs. It would not make sense to build another Tableau.

The difficulty of implementation is not necessarily correlated with business value. In other words, what is challenging in data science may not have business value. Conversely, what is, in reality, straightforward to implement may provide boatloads of convenience to users. Something like a business intelligence tool showing the data the stakeholders already have is often low-hanging fruit.

2. Prototype, Prototype, Prototype… then Socialise It

It is worth spending a few days coming up with a working prototype/proof-of-concept to show your (prospective) users. There’s no need to get it perfect from the beginning — at this stage, things do not have to be complicated. The main aim should be to aptly demonstrate capability and feasibility. Prototyping allows for a few things :

• “Show, don’t tell”- It gives business users, and management the confidence that the team has a sound basis in pursuing the project. It also helps to build rapport with business users.
• Separate Reality from Hype — A working prototype concretises what can be done with the technology. This means a lot to business users who get ‘sold’ on hype-driven projects by vendors who want their business badly.
• Baseline the Dreams — A baseline on which the (prospective) user can dream. In showing them the working prototype, it may trigger off other “use cases” that otherwise the user may not have thought of.
• Tangible Feedback — You can immediately get feedback upfront. People are more likely to offer constructive views on something concrete and tangible, compared to some abstract PowerPoint slide they may or may not understand.
• Dip your Toes into Feasibility Testing — Sometimes, we don’t know how feasible or complex the implementation can be until we try it out.
• Share-ability — Something like a simple web app can be easily “shared”, and allows someone to share it saying “hey, check out what was done by this group, do you think we have a use-case of a similar nature?” In our case, a simple question-answer web app prototype led to 2 more business units wanting in on the solution, much to our delight!

3. Mini Wins are Important!

Innovation is messy, and it rarely starts with a big bang. It is important to identify what some term as “quick wins” to hit the ground running. I prefer the term “mini wins” — projects that are relatively straightforward to implement, and can roll out with minimal risk.

• Celebrate the Little Wins — Having “mini wins” allows the team to frequently celebrate the “little things”. These little celebrations help to build up the team’s morale.
• Build up the Technology — Build base technological modules on which greater things can grow. For example, working on a simple semantic search may lead to more complex search methodologies that can involve things like knowledge graphs to retrieve better insights.
• Team Capability Build-up — Build up the team’s capability — individuals will start to get expertise “tagged” to them.
• You Do Learn Something — In building the “mini wins” and through conversations with users, you do learn something. It could be something like “I wish that this app could do XXX and YYY”, which may spin off into a whole new project in itself.

It is similar to playing a game — you don’t play level 99 difficulty at the start of the game. Starting from level 1 builds up the confidence and skill of the gamer. In my opinion, it is much better to get these “mini wins” than to be stuck in the limbo of a great, big, complex project-to-end-all-projects project.

4. Data is More Valuable than Technology

This was something I realised while working on machine learning projects. Much as I love the algorithms and modelling, the algorithms keep getting abstracted away by :

• Machine Learning Libraries — eg. Keras as compared to Tensorflow. Things just keep getting easier to do. These days, you can even get away with not learning Tensorflow to do basic image recognition / NLP.
• Cloud Services — eg. AWS Textract as compared to a bespoke smart OCR / Information Extraction system. Cloud vendors like AWS / Microsoft and Google have every incentive in the world to make it easier for us to do machine learning by using their web services.

I guess that is the story of tech. You don’t really implement something like quicksort from scratch now anyway. (unless you’re hackerrank-ing for Google/Facebook/Amazon, etc). I do see a parallel to this for machine learning — eventually, it would be abstracted to the point where further abstraction makes no sense.

What was the one thing that would not get disrupted/abstracted away? The answer was clear: Data. Yes, you may argue that the platform on which the data resides will change, but fundamentally, valuable data would always be valuable. It is unlikely that one would suddenly be able to do something more easily with less data.

5. Evaluate the Solution Properly

I have to confess: I am guilty of this. In a bid to quickly push out solutions, we may be blinded by our desire and not see the other initiatives that are going on elsewhere in the enterprise. Remember that if you have thought of it, someone else would likely have thought of it too. Something that you see as a viable project could be :

• Already Done! : Someone could have tried it before and it didn’t work out, or a variant of the idea had already been attempted and it didn’t get any traction.
• A Simple Extension of another Project: There may be other ongoing projects that are a superset of what you want to do. It could very well be the case that what you want to do could be a simple extension of another project.
• Attempted Previously in a Reduced Form: In our case, my Product Manager had asked around and realised that a variant of our enterprise question-answer project was attempted by another group in a company-wide hackathon. This allowed us to obtain a valuable dataset from the hackathon team and saved us some time. Thanks Mook!
• Available as a Web Service: One should also consider the case where what you want to do is already done by a cloud service. One example was our decision to leverage AWS Textract instead of trying to build a bespoke image recognition machine learning model that we would spend a lot of time fine-tuning. In our case, leveraging AWS Textract allowed us to focus on the business value in this project.

So there you have it — 5 learnings that I’ve gone through as a result of working hard on a few enterprise machine learning projects. Hope it helps you on your journey!