Data Scientists and Feature Teams : a brother-sister-like relationship
Since I joined ManoMano as a Product Manager in early 2019, I’ve been working with Data Scientists (DS) in many different set-ups, mainly on recommendation and search engine issues. This allowed our Feature Team (FT) to deliver successive and significant uplifts on our KPIs, which is our proxy to measure whether we better serve users’ needs. It’s been a wild ride full of joy, love, impacts, misunderstandings, arguments and reconciliations.
However, when talking with data scientists and other product managers from the tech scene, I’ve noticed people usually tend to say as if it were natural: “yeah, we have data scientists integrated in the feature teams”. When you dig a bit, it more often ends up like : “We would really want to have data scientists in feature teams, but they are hard to recruit and we are understaffed and somehow we are still struggling to collaborate efficiently”.
One of ManoMano’s strengths is that it is a data-driven company. Thus, allowing closer collaboration with Feature Teams has always been a key challenge in the organization.
For very high-level context, the data science team at ManoMano is mainly in charge of:
- Driving in-depth analyses on advanced and high-impact topics
- Solve user problems thanks to algorithms that end up in production
Feature teams — not so originally — are composed of several developers, a designer, a product manager, a quality engineer and more and more from coming and going data scientists. This is the more interesting part we are going to focus on in this article.
At ManoMano, we think that we found the right mix by working hard together at the beginning of key data project and then being more independent. Here are 2 things I learnt during this journey with our data scientist mates, that are quite similar to the one with my real-life brothers :
- If we work hand-in-hand on a daily basis all-year-long, one or the other won’t be fulfilled.
- If we don’t get along on a regular basis for quality time, we create a shortfall for both parties.
Do not destroy brotherhood by invading your brother’s private life: the run
I am convinced data teams and feature teams should work closely and meet as soon as they can to gather forces and allow more impact. But I also do think the grail of this collaboration is for the feature team to allow data scientists to get total autonomy from them. What does it concretely mean? Developers and Product Managers are owners of website architecture, user experience and performance. Data scientists own the algorithms performance. That’s it. But you won’t easily find someone who is an expert in both data science and software technologies. So when you work on a brand-new data-intensive search or recommendation experience, developers and data scientists must collaborate to maximize impact and velocity of delivery. However, once the MVP is delivered, data scientists and the feature team should not need each other to deliver additional iterations. Data Scientists may allocate more time to understand the effects of the algorithms being used, and should be autonomous to AB-test another one without the FT support. On its side, the FT may have front-end tasks to complete, automatic tests to add, and tracking to implement where the data scientist’s time would be wasted if he/she were involved in these iterations.
In other words, as brothers we’ll meet again for another ride, we both know that. But here are the 2 main reasons why we should not be living together.
We do not share the same temporality
The life of developers in Feature Teams basically consist in imagining, discussing, refining and delivering new and existing features. Developers are much focused on delivering at high pace, while Product Manager has the duty of exploring and understanding users’ needs. Data scientists on the other hand alternate between delivery and exploratory modes, and this exploratory mode (aka R&D) poorly fits scrum methodologies. From my experience, when a data scientist is integrated in a FT for several quarters, it results in frustration about not being able to dedicate longer exploratory times on key problems that would require a longer timeframe.
As with your siblings, you may have experienced long periods without seeing each other, because you or they were busy enjoying an exchange abroad or chilling with a lover. It does not mean you are not going to meet again, but at some point you or they had to take this time to do things without the other, and it is really fine.
We do not share the same reality
Feature teams and data scientists face a major friction when working together, which is they do not use the same tools, do not write code in the same technologies, and even besides the “tech” sometimes do not speak the same language. As an example, the “production environment” is critical for any dev in the sense that if it breaks — especially on a marketplace with high business volume — you are going to threaten the business of the whole company and potentially generate significant losses. So you really don’t want to mess with that. As a data scientist, when you build algorithms that run on a nightly basis, it does not actually threaten the business of the company if it breaks as much as once or twice a week. It could even be completely anecdotic. This can result in tense situations with developers thinking data scientists are neither caring nor rigorous enough, and data scientists arguing (rightfully) their added value is not in maintaining and monitoring APIs.
So like with your brother who would work in a completely different field in the countryside whereas you live in the big city, it is very okay not to agree all the time. My brothers and I rarely agree on political debates. But brothers have a willingness to understand each other, and reunions are always times that create delight in both parts. Just keep in mind that if you spend 3 months non-stop together you might remember why you used to fight when you were kids. All the same, data scientists and feature teams will get tired of each other if they can’t get their privacy back on a regular basis.
Spend quality time together to keep your relationship alive : collaborate in delivery mode
If you never go for a weekend together for years, it’s likely that your relation will go to the dogs and you won’t have much to share in the end. Going on adventures together every so often is a good way to ensure that your relation does not become just a collection of good old memories.
When we first delivered new data-intensive search features on our platform, we spent roughly 6 months working hand-in-hand with a data scientist in the feature team. It meant :
- 4 days a week at the same bench with the feature team
- He attended all the ceremonies, including daily stand-ups and tech refinements
- Aligned objectives with the rest of the Feature Team
- Some pair-programming in PHP to put himself in developers’ shoes
- Presentation (and explanation) of data iterations to the rest of the team
- Beers and lunch together
This allowed us to deliver high-impact features that drastically improved the search experience on ManoMano. During the period I’m talking about, the percentage of users who declared getting irrelevant results when searching on ManoMano decreased by nearly 40%. It also allowed us to discover each other and understand what the most suitable way of collaborating together would be. For instance, we quickly saw that any data scientist — even when he was working in scrum methodologies with the FT — had to keep one or two days a week to work on longer term topics with other data scientists. It also allows them to get challenged by their peers and foster creativity on the solutions they build.
But once these features were delivered, our data scientist started to feel like an impostor in the feature team. He was faithfully trying to stay in the scrum methodology whereas he needed a longer temporality to dig-out on new problems our users faced and crack new solutions. Until he realized that and simply left the feature team.
Was it a failure to have our data scientist leave the FT?
I don’t think so. If we wouldn’t have had our data scientist in the Feature Team, I’m pretty sure the situation would be far different from what we have now:
- The shared achievements that now allow us to break the silo would not exist
- There would be frustration on both sides: from developers feeling data science does not care about their constraints, from the data team feeling developers prevent them from testing their algorithm in production
- The feature could even not be in production as of now
More importantly, it was necessary that after this ride spent together, our Data scientist went back to the data team to work on other key topics for ManoMano. After the weekend spent skiing with my brother when the picture above was taken, we did not plan to move together. Everybody got home, and it is just the way it should be.
Delivering exciting and impacting features together are achievements we have in common and we are proud of. When the opportunity comes, we’ll jump on to do it again as we’ll go for other wild rides with my brother as soon as we can.
Conclusion : spend great times together (deliver impact) and respect everybody’s routine (the run can be asynchronous)
As a conclusion, at ManoMano, Data Scientists tend to be integrated in Feature Teams to deliver specific and impactful features in a time-framed environment. But this is not our goal to have Data Scientists all the time in Feature Teams, as temporality and daily life of each other are just so different.
This way of collaborating is likely not to be our final one, and we are willing to push the collaboration between product and data science much further. For instance we have not cracked yet how we could have our data science team bring more insights to the work of our UX team. It could be our next big thing on the topic to multiply impact of both teams.
On your side, as Product Manager, Data Scientists or Developers, have you experienced other ways of collaborating together ? How do you leverage all those skills to have them get the most impact in your organization ? Please feel free to share your own experience in the comments, we would be happy to discuss and get your best practices !
Special thanks to Pierre Devaux, Pierre Fournier, Grégoire Paris, Timothée Trichet and my data science bros Alexandre Cazé, Yohan Grember, Romain Ayres, Jacques Peeters and Marin de Beauchamp for proofreading this article. And another one to Martin and Simon Desbrières for inspiring me the comparison :)
