Study your domain

Because expertise in data science may never be enough.

Ten people from sales and marketing have gathered in a conference room to hear you speak. You’re the only data scientist, you’re nervous, and you’re reading directly from the slides on the screen. You’re competing for the attention of everyone in the room, but their smartphones are winning that fight. It seems like no one cares about what you’re saying. “Random forest this. Confusion matrix that. Next slide.” You catch someone yawning, and another finally speaks up. “While this is all very interesting, I don’t know what to do with it.”

I’ve been witness to this cringe-making scene on a regular basis, and I’ve played the lead role more than once.

Why does this happen?

Data scientists spend countless hours poring over their data and building models that consider all the critical assumptions and account for the anomalies that make their data unique. With enough time and research, they will uncover new insights, and their models will occasionally yield significant results. But when data scientists try taking their business partners on this journey of discovery with them, they’re often met with ambivalence, apathy, or confusion — as if the data scientists and their business partners are working on entirely different problems.

It doesn’t have to be this way.

If you want to make sure you’re working on the right problems, and if you want to effectively communicate the value of your work to your business partners, you should reserve time for professional development that focuses not only on your discipline but also your domain.

For the successful data scientist, software and statistics make up a necessary but never sufficient foundation of knowledge. The successful data scientist must also study her domain. If she works in the candy bar industry, she will have learned everything possible about the production and distribution of candy bars. If she works in the automotive manufacturing industry, she will have studied the main drivers of assembly line manufacturing and supply chain logistics. As a data scientist in the Cloud+AI Customer Growth Analytics team, I need to know everything I can about our customers’ transition to the public cloud — the economic incentives and total cost of ownership, the pain points and pitfalls, the critical moments that can determine long term success or failure for our customers.

In my experience, there are two proven ways one can become a domain expert in a new field.

1. Spend time with your stakeholders. Learn the details of their business by spending time with them. You could job shadow the product engineering team or the managers in sales and marketing. Your goal should be to understand the decisions they face every day, the resources they have available, and the constraints they’re up against. Once you’ve grounded yourself in the dynamics of the value chain within your company, you’ll start to identify the right application for your skill set as seen through the eyes of your stakeholders. In six sigma training, practitioners are taught to physically place themselves in the center of the processes they hope to improve. Document each decision point, every bottleneck, and every point of failure. If you find a defect or deficiency, ask why five times.

2. Immerse yourself with training, content, and experiences intended for your customers. Our team of data scientists at Microsoft operates in the domain of cloud computing. We are embedded in the very business of designing and building data platforms and machine learning solutions, and we are our own customers. But this is only part of the story. The Azure homepage lists over 100 services across 22 categories, and data analytics and machine learning are just two pieces of the puzzle. Our team invests heavily to keep up with the always-growing breadth of services we offer to meet the expanding needs of our customers. This goes beyond eating our own dog food.

A year ago, I completed the Azure Fundamentals training path on Microsoft Learn — immersing myself in the very training designed for our customers. During this process, I became familiar with Azure Policy, a free service that helps companies govern their cloud infrastructure without stifling developer productivity. With Azure Policy on my map of things to care about, I was able to interrogate the data in more meaningful ways, and I discovered that customers who fail to implement a robust governance framework early in their journey often experience pain from unexpected costs, and they risk migrating the same problems they experienced on-premises to the cloud. As a result of studying our domain, we launched a set of built-in policy recommendations in Azure Advisor that help customers achieve stable and predictable growth, manage their costs, and meet regulatory compliance requirements.

In recent years, most software engineering and data science positions have surfaced in fields other than tech. Every company needs data science, and this is a well-established point of view. But without domain context, knowledge, or experience, every new hire in our discipline will fail to integrate in ways that add measurable impact to our business. If you want to proactively identify useful applications of your skill set, you should spend time studying your domain. Don’t rely on your data science degree alone.

In a conversation I had recently with Fatma Kocer-Poyraz, Vice President of Engineering Data Science at Altair, she had noted that she too questions the value of a data science degree without domain experience.

[The] data science degree is going through what the MBA degree went through a decade ago. People who thought the grass was greener on the other side went for an MBA but were partially disappointed with the outcome in job finding. If you know the use cases, pain points in the industries that you are eyeing and can find ways to apply data science (or MBA); it is great. But if you study these degrees because you think on their own they will help you help any business, the outcome would not be what you expect…

Even if you already have a job, domain expertise will help you keep it. I can hardly count the times I’ve heard that AI will eliminate entire sectors of the job market. But that doesn’t mean working as a data scientist will protect us from losing our jobs. Large tech companies are building out-of-the-box machine learning solutions that can be implemented with very little effort across multiple domains. And as they improve in quality, the ML solutions that depend least on domain expertise will be first to become fully automated. Do you want to recommend the next purchase to a retail customer? Do you want to recommend the next movie from your streaming catalog? Do you want to detect fraud, churn, high spending? Do you want to segment your customers? Just connect the streaming data source and press execute. This is data science without the data scientist — it doesn’t require domain expertise. But domain expertise will help you keep your job when the robots take over.

Next time you find yourself competing against a smart phone for your business partner’s attention, remember that your expertise in their domain will keep you in the fight.

Spend time with your stakeholders and immerse yourself in training, content, and experiences for your customers. Study the domain that you work in. Learn every detail about the environment in which you operate. Don’t ever let your stakeholders reduce you to being “the numbers person” when you’re capable of becoming the resident expert on the topic at hand who also happens to be good with numbers.

Study your domain and you’ll be able to…

• Identify useful applications of your skill set instead of waiting for instructions.
• Communicate the results of your work using language that your business partners will understand.
• Find a good job and keep it for years to come.
• Connect areas of the business that few will see without your unique perspective.
• Make yourself irreplaceable.

You can connect with Matt Wright on LinkedIn or on his blog, 97 Ways.com, a guide for professionals in data science, analytics, and BI.