Why throwing money at data science talent isn't going to help

As a data science company, we have been spending a lot of time with new economy firms, typically digital businesses, who understand and empathize with the difference, applying data analytics can make, to their businesses. Now, I use the term “analytics” with caution here because the mere utterance of the word makes these companies scurry to defend their data-led practices.

Prominent new economy companies, like e-commerce, pride themselves on their data practices but scratch below the surface and you start seeing a different story where the old economy practice of using expensive, out of the box and bloated data analysis software that churn out regular post-facto “reports” and “dashboards” are the mainstay. Of course these companies also employ analysts, who for no fault of theirs, get named as “data scientists” on the back of their ability to find their way around a complex data analysis software.

Until someone from the engineering team wakes up and says, “hold on, we have all this data and infrastructure and if we can a half decent programmer to deploy open source tools we could be doing anything we want and not depend on an expensive software”. Typically this is where things get interesting — both bad and good. Good, because the company is now open to experimentation and looking beyond playing with catch words like big data and machine learning. Bad, because even half decent talent is hard to come by.

“an experienced data scientist is too expensive and an affordable data scientist is too inexperienced”

..and this is a conundrum that throwing money at, isn’t going to solve.

Data science is a deep subject. A computer science graduate with 2.5 years of experience in solving an array of data science use-case / problems, is phenomenally more experienced than a similar graduate with an “interest” in data problems and 2 years of development experience purely because the former has thick experience in applying algorithms and machine learning. The latter is your affordable data scientist.

An affordable data scientist will join a firm and face a finite set of data driven use-cases to help resolve. He will spend 3 to 4 months working on those use cases, gain the experience for him to become the experienced data scientist that the market is willing to pay through its nose for. He will move on to a firm that is eager to give him a 70–80% jump over his current salary, get his flaky experience. And this cycle continues and create a sort of a talent bubble.

Therefore, throwing money has only achieved two major things for you: wasted your money and left your experiments half done. It certainly has not succeeded in delivering any sustainable value.

So how do you tide over the chasm?

1. Take time to evaluate make vs. buy decisions with respect to data science problems.
2. Do not throw money at talent. Respect abilities but importantly respect that the learning curve is steep and has to have a compensation that matches the abilities
3. Invest and empower a data science leader who can bridge the gaps between business expectations and a pathway for data science solutions. Give him a team. Let him experiment.

Its a talent war out there, but splurging money isn't the smartest way to win.

Godspeed.