Machine Learning — No Data Scientist Required?

I’m still digesting all the news out of AWS re:Invent 2018, but boy is there a lot to like if you’re a software engineer and have an interest in machine learning.

Get ready to code

For years, engineers working on data science teams have essentially been working on data pipelines and supporting the work of a team of data scientists who need data to build models. More recently, we’ve seen the role of the Machine Learning Engineer which at most companies is responsible for knowing “just enough” data science to partner with a team of data scientists to get those models to production at scale.

I’ve been playing around with some of the ML services in AWS over the past year and I’ve been impressed. After today, I’m convinced that the relationship between engineers and data scientists is finally changing. Despite the title of this post, I don’t think the role of a data scientist is going anywhere anytime soon. I do think that the common approaches that Amazon (and Google and Microsoft) are turning into “Machine Learning as a Service” are at a point where an engineer with limited ML exposure can take the lead.

Is it really that simple? Well, no. Models still need tuning, and experiments need to be designed and analyzed. For some companies, the ML services will work out of the box well enough to do cool things but an engineer and data scientist will need to work together to get the most out of them. It’s just a much different relationship than before — Partners rather than a supporting role.

I’m even willing to bet that at companies with tight budgets (cough, startups) and a baseline of no data science at all will get worthwhile value out a solid ML-interested engineer and what AWS has to offer. An aside — That’s also a great opportunity for such engineers to get into ML.

Of course there’s still plenty of new ground to break on improving models and coming up with new approaches. There will still be organizations on the cutting edge that need to go their own way and employ a massive team of PhDs. There will also be specific problems that just aren’t possible to solve with what Amazon, Google, Microsoft and others haven’t boxed up (yet). The thing is they don’t have to! They’re covering enough ground to make a dent. We’re finally entering the reality of “Machine Learning for Everyone”, or at least “Machine Learning for Engineers”.