5 Opportunities After Data Science: What To Do If Data Science Isn’t A Fit (Part 1/2)
Life isn’t over, it just gets more interesting. #recoveringdatascientist
Officially, I worked as a data scientist for a grand total of 2-ish years.
Quite short but I learned a lot about myself and what I wanted for my professional career.
One of the biggest learnings was that I enjoyed the software and platform development aspects of creating ML products much more than the actual data science and machine learning work.
And this isn’t an uncommon trend.
Data science enthusiasts spend so much time and effort pivoting to data science, only to realize when they get there that they were in love with the mythical Unicorn Data Scientist story (and the pay) rather than the actual work.
While some data scientists stay in data science and eventually choose to become senior or staff ICs, many more “recovering” data scientists will become data engineers, product managers, technical leaders and strategists, ML engineers, or MLOps/ML platform engineers.
Pictured: Some very famous recovering DS CoughDataEngineersCough .
Nevertheless, I wouldn’t do anything differently. My time working in data science and analytics was incredibly valuable and helped set the foundation for my future role as an MLOps Engineer and then as a Developer Advocate (for data scientists and ML platform engineers!).
Here are all the ways you can leverage your time as a data scientist to move ahead in your career.
Why Would Data Scientists Leave Data Science? (Or Even Data Entirely?)
Data, Reproducibility, Deployment…Production ML Is Hard
Doing data science is hard.
Doing data science well is even harder.
And putting data science to work under real-life conditions? Why, that takes almost 10+ roles to do well.
Some of the challenges of making data science work in production include:
- Need for high-quality data;
- Infrastructure to support machine learning pipelines;
- Ensuring statistical properties of training data match production data;
- Being able to adapt and create new models & pipelines gracefully (& quickly).
A more detailed discussion about the production ML can be found in this Twitter Space between myself, Chanin, and a few other folks.
Always Drink Upstream From The Herd
All jobs are hard though, so are all the reasons I listed previously enough to drive data scientists into a completely different role?
Well no, there’s more.
Some common reasons for the exodus to roles like MLOps and data engineering include:
Data scientists catch everything from upstream because they’re downstream of product, engineering, and politics. (Paraphrase of Goodbye, DS)
Oftentimes, data scientists tend to have very little control over picking their projects. Requests tend to be handed down to them (at best with indifference, at worst with callousness) and can fluctuate with the business.
This lack of autonomy & ownership over the infrastructure & data can sometimes result in work-life balance simulating a roller-coaster ride or the bullwhip effect.
Part of the reasoning for the “large gap between what we can do, and what we are asked to do” (Shakoist, Substack) could be due to the newness of the field and consequent lack of standardization.
The business and product side, while rich in internal organizational expertise, also tends to be poor in statistical methodology and data science domain knowledge. They need to be educated or at least told “no” when proposing infeasible or poorly scoped requests.
This lack of data and scientific literacy can also contribute to:
Terrible Management & Insane Projects (Goodbye, DS)
As in, being given infeasible or poorly scoped requests expecting magic to happen.
Readers of both “Goodbye, Data Science” and “Why Business Data Science Irritates Me” seemed particularly taken by (or emotionally validated?) by the discussion around how there’s no reward for bad data science work and sometimes failure is even rewarded beyond good data science work.
What then is the incentive to do incredible work?
The lack of accountability was also a big theme in both posts, specifically the prevalence of “decision-driven data” instead of “data-driven decision” (Goodbye, DS) or as Shakoist puts it, “providing insights suffers from a principal-agent problem, because giving someone true insights into the data, will not be rewarded as well as giving someone nice intuitive insights.” (Shakoist, Substack)
Some former data scientists, if you were to ask them to describe their experiences working as “business data scientists” might go so far as to recite Will Roger’s line “There are men [➡️ PM’s] running governments [➡️ AI projects who shouldn’t be allowed to play with matches [➡️determine data science roadmaps or resourcing].”
Suffice it to say, go read the blog posts for additional details.
For now we’ll assume that you’re fully bought into that for some, data science isn’t the first or the last stop on the career train.
Your Options After Data Science
Let’s jump into your options!
Option 1: Move laterally into management, lean into strategy, product or even people ops
As a data scientist, you learn and develop important skills such as:
- Data analysis with SQL and scripting languages like Python or R and visualization libraries like Seaborn and Plotly;
- Communication with business partners and stakeholders, as well as domain and business model expertise;
- Experimental design and testing best practices (as well as gotchas) to evangelize;
- Model development and evaluation.
You can leverage many of these skills, directly and indirectly, to improve business operations and provide data-driven insight into key areas like product, operations, finance and even revenue operations.
“But Mikiko, isn’t a data scientist supposed to do that anyway?…
provide business value through automation and predictions?”
Technically speaking, yes.
But data science tools, techniques, and methodologies (which are largely derived from statistics) as well as data analytics have become so widespread, that there are many non-data scientist or engineer roles where being data-driven is a must.
And although consulting firms like Mckinsey have tried to make “data translator” a job category and title (it’s really not and they need to stop), there’s no doubt the ability to wield data effectively to answer strategic business will accelerate the careers of non-engineering or non-data scientist roles.
In many cases, the transition would be relatively easy because of their closeness to data analytics and business intelligence. A technical candidate could come in at a more senior level due to their technical expertise and capabilities.
Careers where data literacy, experimentation design, forecasting, and hackery skills come in handy include:
- Product Management
- Growth
- Marketing (sometimes seen as a part of growth)
- Operations
One of the biggest differences former data scientists experience when they move into these roles is they have more sway and impact on how projects get determined and run.
One reason why this happens is the difference between “formal authority” and “influence”.
Formal authority is what you’re endowed with due to your title and the responsibilities (and scope of power) it comes with.
For example, a product manager typically has broad latitude to determine what types of projects meet the company’s current objectives, plans the project with other teams like engineering, and helps coordinate the resources and moving pieces (including setting up experiments, user studies, etc). Their authority comes directly from the role itself.
However other sources of power can fall under “influence” including:
✅ Relationships — Having strong relationships with teammates & business partners;
✅ Organizational Understanding — Possessing knowledge of the inner workings and concerns of the company;
✅ Expertise — Developing a toolbox of knowledge & experiences that can be leveraged & utilized by your team & partners.
As a data scientist moving into a role like product management or growth, you would be bringing a significant amount of Expertise. And if you make an internal switch then you’d also be benefitting from (or not suffering from a lack of) Relationships and Organizational Understanding.
Option 2: Move horizontally into an adjacent discipline
Some adjacent disciplines to data science include ML engineering, MLOps, and data engineering.
Although additional upskilling will be necessary to switch into these roles as they’re not exactly a 1:1 mapping, prospective candidates aren’t starting from zero.
Candidates go into these roles with a deep appreciation and empathy for the kinds of challenges data scientists face every day as well as the impact of tooling choices on the ML lifecycle and workflows.
In many cases former data scientists that switch into these roles feel a deep sense of relief because they’re:
- Closer to the business;
- In positions where they can actually influence architectural decisions (both good and bad);
- Platform and engineering cycles tend to be more decoupled from the vagaries of KPIs and product whims than data science (especially since in some companies, data science teams report into Product!);
- Success is less vague & more concrete (for example, if models fail tests or in production, that’s more obvious than “the model didn’t increase revenue by 10% even though the model is 30% more accurate” whereas a data pipeline failing is definitely bad).
If you’re interested in learning more about pivoting to MLOps, check out my blog post about what an MLOps engineer does & what my job search was like (part 1, part 2).
🤖 What An MLOps Engineer Does 💻
📆 And What The Week Can Look Like
If you’re interested in learning more about data engineering, check out some of my friends Joe Reis, Matthew Housley, Mark Freeman, Shashank Kalanithi, Zach Wilson, and Seattle Data Guy (some of whom coined or popularized the “recovering data scientist” moniker).
Interested in the other ways to leverage your data science career for money & profit?
Check out part 2!
And don’t forget to follow me at:
- LinkedIn: https://www.linkedin.com/in/mikikobazeley/
- Youtube: https://www.youtube.com/@Miki_ML
- Twitter: https://twitter.com/BazeleyMikiko
- Mastodon: https://data-folks.masto.host/@mikiko
