Ways of Working in a cross-functional data science team
It can be tricky to fit agile ways of working in a data science team — perhaps even harder when there’s a mix of data scientists and software engineers!
In my previous blog post, I discussed tips and tricks in doing this in a purely data science team. Since then, our team has grown to become a truly cross-functional team, and we have adapted our ways of working to suit our changing needs. In this post I’d like to share some insights we’ve gained along the way!
About our squad (now called Kimchi!)
We started off as a small team of 4 data scientists (DS) and 1 product manager working on Menu-related data products, but we have since then been joined by 5 software engineers (SE) and 1 product designer. We are a feature team, focused on developing Menu Planning products in Gousto, ranging from the menu planning algorithm to the menu planning front-end tool. These are used by the Food team at Gousto to plan menus automatically rather than by hand!
So … How can DS and SE work together effectively?
1. Avoid silos using shared goals
Often, DS and SE are forced together in one team, but they work on completely different things. Then they start to work separately, stand-ups are not relevant, and they become siloed. The most effective way to stop this is to plan and schedule work so that Objectives and Key Results (OKRs) are aligned as a team. Working on the same goals keeps the team cohesive — it’s important to have a shared vision, and to make sure everyone knows how they can contribute to those goals. For example, one quarter we may aim to release the feature of making recipe swaps on the menu easily. DS would work on the algorithm that suggests the best swaps, and SE would work on surfacing these in the tool.
2. Have Epic owners in each function
We split our projects into epics, which can be broken down into smaller tickets. In a bigger, cross-functional team, it’s useful to have epic owners for each project, one representative from each function. An Epic owner is responsible for ensuring that any estimates are up-to-date, all tickets are refined and scheduled, liaising with the other Epic owner for alignment, and updating the rest of the team on important decisions.
3. Be flexible around agile ceremonies
Feel free to mould some agile ceremonies to suit each function’s need! Some may need to be held together, some at a different cadence — experiment and see what works. This is what worked for us:
- Retrospectives: We have a squad wide retro every two weeks, to discuss topics related to SE/DS features we work on together and any other more squad-wide issues. We also have function specific retros every two weeks, in which we can discuss DS specific issues only (and same for SE).
- Sprint Planning and Refinement: We hold these separately. The product manager attends both planning sessions, to make sure our work is sequenced correctly. The epic owners are responsible for ensuring we have refined tickets that we require in the next sprint.
- Sprints: DS have short 1 week sprints, as it gives us greater flexibility given the higher level of uncertainty in data science, as highlighted in my previous post on Making Agile work in Data Science. SE have a separate 2 week sprint which serves their type of work better.
- Daily stand ups: Done together, with brief updates when we’re starting a new sprint, to update each other on our key goals for the sprint so we each have context.
The above ensures we only attend ceremonies that are relevant, but we always stay updated.
4. Be clear on roles and responsibilities
Make sure to define roles and responsibilities as a team, to iron out any confusions of what part of a project lies with which function. We found this out the hard way when we scaled our team rapidly, and found it hard to make key decisions and know who to involve in what meeting. After having a session to define these, we could reach decisions more effectively.
5. Revisit Ways of Working across different project types
The nature of data science is that we often have different project types; one day we might be doing some exploration, another day we might be writing production level code, which would affect SE’s systems. It’s useful to define different ways of working and coding standards for these. For example, we use rigorous practices for writing code that affect SE, such as hitting a certain test coverage, seasonal refactoring, more reviewers of pull requests, and more detailed user stories and acceptance criteria in tickets. For Discovery/Proof of Concept projects, in which we explore whether a model may be viable, we’re more flexible. Setting these ensured we still deliver work on time but do not compromise on high quality when needed.
6. Have some socials!
Something we can’t ignore is team bonding and spirit! In a smaller team it is easier to maintain great team bonds — chit-chat is much easier in a stand-up of 5 people rather than 11. In a bigger team, ensure to put in the extra effort in for a close-knit bond in the team. We have a weekly ‘Wheel of fun’ where we pick a random lucky winner each from SE and DS, provided with fun topics to talk about, a fun ‘squad day out’ doing a treasure hunt around London, and of course, many pub meet-ups. I’m proud to say that our squad had the best representation at the Gousto summer party!
Conclusion
Overall we have worked to retain the flexibility that our previous ways of working offered, such as our 1-week sprints, but with more structure that enables great collaboration between different functions. I’m proud to say we still remain one of the happiest squads, with top squad health scores in Gousto!
