Evaluating ML Opportunities, Part 3
This is the final part of a three-part series laying out questions you should ask before you start a machine learning project. In Part 1, we considered prerequisite questions to ask yourself and your team before starting out. In Part 2, we looked at project-level questions.
Once you’ve identified your potential project(s) and you’re ready to move forward, you need to assess if you have the right process to pull it off. After all, you don’t just need project ideas. You need the means to carry them out. So let’s get to the final round: process questions.
Do You Have a Plan to Get All the Way to Business Impact?
You need to understand how each step of the process will lead to you business impact. If you’re not capable of making a plan to ultimately get machine learning into production and then measure and optimize its impact, there will effectively be no business impact.
This question doesn’t need to stop you in your tracks — you don’t need a perfect plan, and you’ll likely need to make changes as the project moves forward. But if you have no idea how you’ll take a project to the finish line, you might be better off picking a different project. You don’t want your technical team to present you with a working model only to be left wondering what the heck you’re supposed to do with it.
Do You Have What It Takes to Get Through a Potentially Messy, Uncertain Process?
ML projects require resources, support, and persistence. Data prepared with the best of intentions can still have problems. Top data scientists may struggle with a project for weeks before having an “a-ha” moment. ML projects do not always go according to plan. Ask yourself whether you’re willing to go through that potential struggle, and if your organization is capable of supporting it. That support also has a financial cost, which leads to the next question:
Is Your Budget Ready for What This Will Actually Cost?
If you’re new to ML, be especially careful with budgeting decisions. Projects may be more complex and take longer than you initially anticipate. Data wrangling issues might add weeks to your timeline. Your deployment process could take an extra month due to unforeseen circumstances. Will your budget support that?
Of course, the goal is for ML’s benefits to be far greater than the costs — and given ML’s potential, that is possible. But that doesn’t change the economic fundamentals: you want to try to get the most bang for your buck.
How Long Is It Going to Take to Get a Proof of Concept?
It can be important to show some results quickly to obtain and maintain momentum. You don’t even have to pick low-hanging fruit at first — just demonstrate that it can be picked. Say you want to analyze emails — how long would it take to build a POC that can analyze three?
How Long Is It Going to Take to Get to Business Impact?
While there may be exceptions, most ML projects should not stretch out for years. Ideally you want to get something into the real world quickly to make sure it works — and you don’t want to spend a lot of time building something only to find that your business has moved in a different direction.
How Ready Is Your Data?
This question involves a deeper dive into your data than you took in the prerequisite section. Examine more closely whether you really have the goods.
In order to do a machine learning project, you need data that is Accessible, Sizeable, Useable, Understandable, and Maintainable (See “Assessing Your Data Readiness for Machine Learning”). Will your data be flawless? Almost certainly not. But at the very least, you should deprioritize projects for which the data is clearly not ready, and more heavily weight projects for which the data seems solid.
Start with testing the accessibility assumptions you made in the prerequisite stage: just try to get your hands on some of the data you’re thinking about using. Does the data exist? Do you have to talk to some other group where you don’t have explicit authority in order to get it? It may be harder to get data than you think!
Do You Have Internal Support?
Machine learning is a team effort. You need budget and resource approval from your leadership. You need data from teams that you might not be a part of. You may need frontline employees to use the solution you build, even if that means getting over initial fears about their jobs.
You may not have support from everyone, but try to go where the support is greatest. Machine learning is hard enough without active resistance.
Series Summary
If you don’t have an ideal answer to every question in this series, don’t sweat it. The real goal here is to help structure the discussion as you plan with your team — and to make sure you don’t start your machine learning journey with obvious blindspots.
Weight these questions in a way that works for you. And be honest about the answers you do have— a candid conversation among technical and business principals could save your organization months of time spinning wheels on a project that never really mattered in the first place.
Evaluate your machine learning opportunities wisely, and you’ll be on a path to maximize your chance of successfully making business impact.
James Kotecki is the Director of Marketing & Communications at Infinia ML, a team of data scientists, engineers, and business experts putting machine learning to work.
