How to collaborate with Data Science teams on delivering features to production? 5 Tips for Dev Managers

The value of integrating Machine Learning algorithms into SaaS products has become much clearer in recent years, especially now with the surge of generative AI and its public availability through open APIs.

Efficient work between the development teams (who bring these capabilities to production) and the Data Science (DS) teams (who are usually the owners of these and other powerful ML/AI capabilities) is important now more than ever in order to bring value to real users in production.

Productizing DS-powered features can be challenging and even frustrating for development teams and their managers, as DS practices might seem to break common development workflows and best practices.

As a development manager, you might be tempted to treat the integration with a DS deliverable (trained model, draft algorithm, or pseudocode) just as you would any other integration with a component developed by another team: we’ll define the boundaries between the components, work on these components independently, and eventually integrate them. But this is not necessarily the right approach.

In reality, the DS deliverables — while extremely valuable — are often not a production-ready component we interact with, but rather a blueprint for building production code. This means that when the time comes and we start integrating it into the product, we’ll find ourselves going back and forth with the DS team until things are working. It is important to understand that the work of those two teams is not a relay race, it’s more like a dance: each time someone leads, and each participant has their solos. While dancing, you must be aware of both yourself and your partner’s challenges and be there to support each other. You both need to be at your best for this show to be a success.

Below are 5 tips for managing such projects with DS teams.

1. Be involved in the DS work from day one

Your engineers don’t need to be in the loop at this stage, but you — as their manager — must be there. Read the relevant design docs and ideas, and if there is no written material, ask for a meeting to understand the planned effort and its goal. Every DS work should have a well-defined goal! Make sure you understand what’s planned and how it achieves it.

Since DS projects might provide partial results or end up being entirely scrapped (e.g. if there’s simply no correlation where you hoped to find one), you need to get a good feel of how close we are to a solution. How do the DS engineers feel about the odds of success? Do you agree with their estimation? Where are we in the process — is it at the very beginning of the proof of concept or steps away from completion? Are we stumbling around in the dark or is there some theory we are trying to prove?

This early involvement will give you the critical background for having meaningful discussions and understanding concerns throughout the next phases of the project.

2. Provide needed engineering support even before the development is officially started

During data analysis and algorithm development, the DS team might face many engineering challenges that are not algorithm-related (e.g. loading the data from one store to another, accessing internal data APIs etc.). Make sure your team’s engineering expertise is utilized to unblock the critical path efforts in the DS team.

Assign a dedicated engineer to attend the DS team’s daily meetings and provide the needed support.

Having even one engineer involved in these early stages allows the DS team to focus on their strength and really make the process efficient. It will also introduce an additional team member to the DS work for a smoother transition later on.

3. Don’t write even one line of production code until the DS work is summarized

The summary doesn’t have to be final, but some clear result needs to be presented and agreed upon.

It doesn’t mean you cannot assign the lead engineer to start documenting, add notes, start discovery or even write an HLD (High-Level Design) whenever you want, but coding too early might tie you to implementations that might turn out to be incompatible with the eventual algorithmic solution.

4. Involve the DS team in your development milestones

Once development does start, have a DS representative in your kick-off meetings and any other milestone meeting. Share your HLD, open questions, and timelines with the DS team — it is critical for getting feedback/concerns as early as possible.

Make sure both teams are aligned on the following:

• All APIs between teams should be clearly defined and documented
• Keep a clear separation between the algorithm delivered from DS and your implementation concerns — the algorithm should be decoupled and have clear boundaries and definition-of-done
• Agree on a clear KPI for the algorithm’s quality, and monitor it. This is a challenging and critical step for understanding and reflecting on the value for ourselves and for every level in the organization.

5. Last but not least: Reaching production availability is the finish line for both teams!

DS teams have their own cycles of trial and error, which might seem endless to delivery-oriented development teams. This is needed and beneficial for the research phase, but once development has begun, help the DS team remain focused on delivery.

Make sure the relevant checkpoints are being tracked and marked as done throughout the entire development process. Stop and provide mitigation if checkpoints are missed, and take decisions that will allow you to complete the lightest end-to-end flow first.

You as a development manager responsible for delivery must find a way to keep your development cycles and deliver features to production on time.

Summary

working with DS teams is all about collaboration — utilizing each team’s expertise while not forcing one team’s practices on the other. As a manager, it’s your responsibility to get the right people involved at the right time and facilitate effective communication between teams whose vocabularies might be slightly different. Meeting that challenge can produce quick delivery of valuable features for your users.