80% of all machine learning engineers work at Google or Facebook
Here’s how the rest of us can do meaningful things with artificial intelligence or machine learning since all of the current AI experts are already working for Google and Facebook.
First and foremost, I’m going to argue that you don’t necessarily need a PhD to be an AI expert.
In order to get a PhD, you need to have completed a doctoral thesis, which is a lengthy research project usually mixed with additional education, done with close supervision by another academic. This is, in general, a good thing. We want research, we want knowledge, we want people to get PhDs.
In the world of machine learning and artificial intelligence, PhD work often centers around developing new algorithms to accomplish more and more computationally complex things. Perhaps if we can boost the accuracy of a face detection algorithm from 97.3% to 97.7% we’ll have accomplished something worthwhile.
In the world of practical AI such intense research on algorithms and new methods of computation take a long time to become useful, because they are often not optimized for business (nor should they be).
The real point is that you don’t need a PhD or a data scientist (or both) on your team in order to deliver something of value with machine learning.
Today, right now, at this moment, there are a plethora of APIs and machine learning tools that can deliver a tremendous amount of value, all you need to do is follow a set of best practices that doesn’t require 8 years of research to pursue:
Focus on the problem you’re trying to solve
This is something entrepreneurs, developers, and founders do really well. We don’t have time to do something for the sake of doing something, we have to solve a real world problem. Machine learning is great for doing that, but only if your problem is well defined and isn’t just “predict the stock market”.
Always start small and work your way up from there. For example, maybe you want to tag people in photos your customers are uploading to your platform, or flag user-submitted news articles (ahem…Reddit) as opinionated or biased.
Both of these use cases can be (pretty-easily) accomplished today with off the shelf APIs and tools that don’t require any machine learning engineering knowledge whatsoever.
Have good data
If you’re training a face recognition system to recognize your customers, or trying to classify news articles, just make sure your training data meets some simple criteria:
- You have samples from content your model is likely to see in the real world
- If you’re classifying things (like opinionated news articles) make sure your classes are balanced, i.e. have an equal number of examples in every class
- You’ve got enough samples (not strictly necessary for face recognition)
Spend your energy on good deployment and devops practices
Don’t worry about the algorithm or accuracy as much as you worry about deployment and scalability. Machine learning should run like any other software in your stack; it should be fault tolerant, lightweight, and owned/managed by you (not outsourced to someone else).
You might get some false positives or false negatives, you might have some variation in performance, and you might see errors. The best implementations of machine learning are not so because of accuracy but because of how they handle these exceptions. When Apple Photos gets someone’s face wrong, it lets you change it, and thereby trains itself on that change to be better.
You can and should do this to.
Pick tools that fit your need
Here are some of my favorite DIY/out-of-the-box/no-data-science-required tools:
Machine Box — runs on premises, lets you train with minimal or no data at all, start with pre-trained models, tune the models on the fly, or build your own. Very simple API. (full disclosure: this is my company so I hope you pick this one)
Google Vision API — runs in the public cloud, has some training capabilities, extremely accurate image recognition, medium to simple API.
Azure Vision API — also runs in the public cloud, some training capabilities, massive pre-trained celebrity recognition database, simple API.
AWS Rekognition — public cloud endpoint, good with face detection, API is not as easy as Azure but still pretty easy.
How do you get to Carnegie Hall?
Practice! Practice Practice! This is your data, your use cases, your niche, your market… you need to experiment. There’s no grand machine learning model that will solve all the world’s problems. The best models are the smaller, finely-tuned models that run JUST on your dataset.
Spend some time cleaning your data, make sure it is a really good subset of the data you plan on having your machine learning model make predictions on later, and for god’s sake don’t expect it to be perfect the first time around.
The more iterations you go through, the more models you train, the more you experience, the more likely you are to see success. Face recognition not doing what you want? Try altering the training images or using freeze frames from your own data (assuming its video — thats where face recognition usually gets tricky). Accuracy only 90% on detecting biased news articles? Find more articles! Or deploy your model and crowdsource the answer from your users!
You can do this! PhD, MBA, MSC, BA, BS, GED or not.