Why Data Scientists Need to Ask “Why”

The other day we had a conversation with a bespectacled senior data scientist at another organization (named X to protect the innocent). The conversation went something like above.

Many of us have had similar conversations with people like X, and many of us have even been X before. Data scientists, being curious individuals, work on problems for the sake of doing something interesting, fun, technically challenging, or sometimes because their boss heard about “big data” in the Wall Street Journal. These reasons are all distinctly different from trying to solve an important problem.

This can be hard for data scientists, because some important problems don’t strictly require a data scientist to solve. There are whole departments of experts already working on these things. It is increasingly the case, however, that data can be used as an extraordinarily valuable resource to help solve age-old, time-tested business problems in innovative ways. Operations? Product Development? Strategy? Human Resources? Chances are that there are some data out there now, or that you can collect, that can help change your organization or drive an exciting new product through new insights.

How to solve real problems with data

To tap this increasingly abundant “natural” resource, however, a data science team must:

1) Learn from business domain experts about real problems
2) Think creatively about if and how data can be used as part of a solution
3) Focus on problems that actually improve the business.

Going in any other order is a recipe for disillusionment about big data’s true potential. Starting with a real problem instead of starting with some interesting dataset often leads data scientists down a completely different — and much more fruitful — path.

The take home message

Be ready. The answers to this deceptively simple question may surprise you, take you into challenging, uncharted territory, and inspire you to think about problems in completely different ways.

Adapted from our original post on O’Reilly: http://radar.oreilly.com/2013/04/why-why-why.html