So You Want to Become a Data Science Manager?
How to think about the IC to Management transition
Motivation
I recently came across a great book called “The Manager’s Path”, written by Camille Fournier. In this book, Camille describes the possible career paths of a technical individual contributor (IC) as one continues to grow in her career. I find this book illuminating because it touches on a rather common topic in the tech industry — “Should I transition from IC track to management track? If so, when is the right time?”. If you have been working as an IC for the past few years, chances are, you have pondered on this question already.
Given that this topic is discussed so extensively in engineering (see here, here, and here) and design (see here, here, and here), you would think it is all figured out by now. However, for a field as new as data science, I haven’t seen a whole lot of discussions about this topic just yet. After working in the field for more than half a decade, I have developed my own perspectives on how to think about this topic, so here we go!
First, My Experience
I am not a data science manager, nor have I ever been one. During the past six years of my data science career, I have definitely considered becoming one, and have even been encouraged to. However, like any other important career decision, I am always convinced that this transition, if made, should be a deliberate one. Far too often, I have seen ICs turned managers for the wrong reasons, which made them and their teams suffer.
Recently, I worked with our data science leadership team to build the first data science tech lead pilot at Airbnb. Through various discussions with ICs and managers, we debated the pros and cons of staying on the IC track v.s. going into management. These conversations not only helped me to launch the pilot but also helped me to contextualize some of my own, often implicit, preferences.
On the one hand, I have always aspired to become a hands-on data scientist, and enjoy doing so from the trenches. On the other hand, I sometimes experience the challenge and inertia of leading without authority. Some people say that there’s diminishing returns for remaining as an IC for too long, and other managers have lamented how far they are drifted away from technical work. While all of these might ring true, I wanted to dig deeper to my inner compass.
The Framework From The Book “Drive”
A few years ago, I came across Daniel Pink’s bestselling book “Drive: The surprising truth of what motivates us“. In his book, Daniel talked about three important ingredients that make our works motivating:
- Purpose: the goal to do something meaningful
- Mastery: the urge to get better at stuff
- Autonomy: the desire to be self-directed
Since then, I have been using his framework for evaluating projects, jobs, or more generally career opportunities. In the next couple of sections, I will again use this framework to talk about the pros and cons of being an IC v.s. becoming a manager.
Purpose
In a well-run startup, everyone knows exactly what the company is trying to achieve — the cost of communication is low, and decisions are made swiftly. As the company continues to grow, communication and decision making become more localized, and misalignments could occur among people who do not communicate effectively. This is why we often hear the phrase “the first casualty of growth is communication”.
In a sufficiently large company, the purpose of managers is to clearly understand the goals of the company and consistently reinforce the company’s vision in order to align teams and individuals to work toward that vision in a fulfilling way. Very different from the role of an IC, a manager’s responsibility is to synthesize information, to provide context, creating the right environment, and do so consistently to create trust.
As described in Julie Zhou’s “Unintuitive Things I learned About Management”, one of the unintuitive things she learned as a manager is that it’s a lot more important to focus on the “why are we doing X?” rather than “how are we going to execute on X?”. The “why” is built on information sharing, trust, and alignment.
This is why management is a career change, not a promotion (see here). If you are ready to serve the mission by amplifying others, by coordinating with partners, and living by the company’s vision, then the managerial path might be a great fit for you. If not, then you should think carefully if you are serving the right purpose.
Mastery
This leap and transition from IC to management track is far greater than just a title change. It’s a fundamental shift of one’s core identity and how one functions. Your technical skill is no longer your most valuable currency, trust is. Code is no longer what you produce, it is relationships. Finally, be prepared to spend a lot of time talking, meeting, and writing Google docs with other people.
To adapt to this new reality, you need to develop soft skills with a higher bar — communication, active listening, empathy, … etc. Many ICs, especially the most effective ones, find this transition challenging, because getting better at these skills is often a lot more amorphous than say, writing more code or conducting more rigorous analyses. The path of mastery is fundamentally different.
In addition to learning new skills, you have to learn how to let go of skills that you have already mastered. You need to know how to delegate effectively and not spend too much time dwelling on execution. If you cannot change this mindset, you severely limit the potential of your team and their output. Paradoxically, the right behavior of a good manager is not to try to be the most technical person in the room.
When you learn new skills, you put other skills in the background. This is natural given the shift of focus. Having said that, this combination of needing to let go of the skills you have already mastered and expanding new skills beyond your comfort zone is a bitter pill to swallow. But, if you are ready to embrace new challenges, this can be an incredible opportunity building foundational and transferable skills.
Autonomy
Finally, as a manager, your schedule will also look fundamentally different. On this topic, I found Paul Graham’s essay “Maker’s Schedule, Manager’s Schedule” extremely relevant. Paul explains that “makers” need concentrated and uninterrupted time to create, usually in half-day chunks. Managers, on the other hand, typically jump in between meetings in hourly chunks.
As a manager, you largely need to keep your calendar open so you could have 1:1s, context sharing, communication, alignment, or even fire fighting. This schedule is a pretty big departure from what ICs are used to, and mixing the two schedules can lead to very poor productivity.
If you are someone who deeply subscribed to the philosophy of deep work, you probably strongly prefer the maker’s schedule. As a manager, you need to be more creative and creating the equivalent of deep working time for yourself. I have heard stories of how CEOs of large companies typically block off time so they can “think”. Deep work in the context of managerial work is possible, but you need to create that space for yourself.
The Concept of a Tour of Duty
Reid Hoffman, cofounder of LinkedIn, proposed the concept of a Tour of Duty in his 2014 book “The Alliance”. He characterizes a tour of duty as: a commitment by the employer and employee to a specific mission of finite duration. Transitioning to management, unarguably, will be a big shift, but I think when looking through the lens of the concept of a tour of duty, it is actually quite liberating.
For young working professionals (myself included), your first rodeo in management does not need to be a permanent one. A much healthier and liberating way to look at this transition is to think of it as a tour of duty. Think of it as a mission that is tailored for you in which you can learn orthogonal skills that are different from the technical skills you already have, and think of it as another way for you to provide value to the organization.
As you approach the end of the tour, you can work with your manager to analyze if such a tour is the right fit for you and for the company. It might not work out for you, but even then you have eliminate this career path, which is very useful for career planning. But if you do like it, new doors are opened for you.
Summary, Not Conclusion
It is probably clear by now that I am not being very prescriptive in this post (I myself am still trying to figure out). The lesson I am trying to preach, perhaps, is to recognize that your goals and aspirations will change over time, and that’s completely normal. The important thing is to make deliberate and informed choices that are aligned with your inner compass as transitions are made, and think from first principles on why you want what you want. It’s hard, but it’s worth trying.
Are you ready to become a data science manager?
