5 Principles for Applied Machine Learning Research

Over the past three years, I’ve spent >50% of my time thinking about what the applied research teams I’ve been part of should be building, and how. This post is about some of the challenges I’ve faced helping to organize applied machine learning research in a hyper-growth setting.

These observations are subjective and overfit to my personal experience; if you’re leading teams at Google Brain, this blogpost is probably not for you. If you’re in a startup which is pre product market fit and seems to be spending a lot of money on GPUs, listen up — here are some truths you can’t find on Stack Overflow.

1. You will build useless stuff, so build it fast

It’s tempting to build palaces of code, with turrets, fancy windows, and spiral staircases. Instead, you should try and build tents: light, simple, and easy to move once you realize you’re in completely the wrong place. Photo by Şahin Yeşilyaprak on Unsplash

You will end up building useless stuff. In a highly uncertain startup setting you’re guaranteed to write lots of code that you’re going to need to throw away.

This is ok, because it’s by writing this code that you learn which bits are useful and which bits are not. Some models won’t work; others will work, but not provide business value; others will be superseded by some amazing open source implementation a month after you’ve finished writing them. Your job is to make sure that the code you now need to throw away was written as quickly as possible.