Why Scrum and Story Points Are Not for Everyone
A little bit of history…
I will not make a long presentation of myself but here is what I can say: I have been keen on the SCRUM methodology for a long time and I can’t (could not) imagine working without Story points. For me, story points are an essential metric for a (scrum) manager to predict a project, to measure the performance of the team… Ask my former colleagues at PMU, I even organized a workshop for them to understand the concept of story points that involved atoms and Star Wars ships (but that’s another story 😉). I even actively participated in putting in place SAFe in my former company. When I started at Doctolib, my first mission was to launch a new Feature Team: organize the way we work, find our pace and of course deliver new features 😄 So I had no doubt: we had to put in place SCRUM, use story points, evaluate our velocity according to the two or three first sprints and GO ON!
The journey to change my mind…
I put in place all the necessary tools to give insights to the team:
- A scrum board of course
- A team dashboard with several gadgets like Sprint Health, Burndown, …
The team contributed of course — no decision was taken without them. They defined their own story points scale, the pace of the sprints, the amount of stories versus bugs and/or technical tasks, … Together, we defined the amount of points we can achieve per sprint and we ran our sprints like this.
We succeeded in delivering our features (almost all the time) but to me, we had 2 main problems:
- We tended to try to stick our feature into one sprint: if a feature didn’t fit into a 2-weeks sprint, we either reduced the scope or lengthened the sprint. It was frustrating for both our PM and for the developers.
- I found the story points system to be inconsistent: sometimes we tackled a 3-points ticket in 2 days, sometimes it took us one week…
After three quarters, I felt a routine, a loss of impetus of our pace. But I couldn’t really put my finger on why.
And during an Engineering Managers sharing session, a fellow colleague (hello Yoann 👋🏻) presented their experiment of “no estimate” (in fact, no story points) into their team. And it made me think: what if SCRUM and story points were not right for my team?
I didn’t want to impose my vision to the team so I organized two “let’s improve our pace” workshops with the team:
- The first one to figure out our paint points,
- The second one to find some solutions.
Even if I suggested moving to Kanban methodology, it has been voted and approved by all the team. So we decided to give Kanban a try. We had 2 weeks to adapt all our rituals and tools to be ready to start the new quarter (and the new year) with serenity.
How did we adapt our rituals for Kanban
Several questions appeared, and the most important one: how are we going to plan our features, to give some insights to the PM on when a feature will be ready?
First of all, we decided to NOT use a “no estimate” method. We don’t use story points any more but we do have an estimation: every ticket must be ready to be merged in 2 days — it can be less but not more.
To plan a feature (epic), I put in place a timeline into Jira:
As I know that every ticket should be merged in 2 days, I set for each ticket a duration of 2 days. I know how many developers will be here per week (let’s say I have N developers in the team), so I always plan N-1 tickets in progress in parallel. By doing that, it gives some space to the developers for pair programming, tech scopings, discoveries, bugs (yes it happens sometimes 🙄), technical tasks, … (to be clear: I say “I” but in fact, I do this planning with the tech holder of the feature). With this tool, we have a pretty good idea of the delay we need to develop a feature.
Well… now that we have the plan, how are we going to follow the progress?
Of course, I put in place a Kanban board: not difficult, I set the exact same workflow for the “in progress” tickets.
So came the question “how do we manage the backlog?”. With Scrum and story points, it’s easy: tickets with no story points must be groomed, tickets into the current sprint must be done. Here what we decided:
- The tickets to groom are in the status “New”
- The tickets we can prioritize are in the status “Ready”
- The tickets prioritized are in the status “To do”
We review the “to groom” and “to prioritize” tickets every week during our backlog grooming in order to always have 2 or 3 weeks of work in our “to do” column. The grooming is not there anymore to estimate the number of story points per ticket but to check if the ticket respects our scout rule, ie. being mergeable in 2 days. This grooming session replaces the former sprint planning. Apart from that, we kept the same rituals we made with Scrum: daily, retrospective and kick-off of a feature (meeting during which the tech holder presents their scoping and the team grooms the tickets).
To follow the progress of the team and to replace the tools I used to use, I put several tools in place:
In the team dashboard:
- I replace the “sprint health” by the number of tickets per status with a ⚠️ emoticon when there are too much tickets in a status
- I replace the velocity chart by a graph showing the number of tickets resolved per week
In the Kanban board:
- I set a color rule to highlight the late tickets and/or the tickets not updated in 3 days,
- These late/not updated tickets are also automatically put into a specific swimlane of the board to easily see them and discuss about them during the daily.
And of course (or not), I finally found my courage to understand the control chart and it’s in fact a quite good indicator of the velocity of the team: we went from a rolling average of 4.1 days during the first year to a rolling average of 2.3 days once we moved to Kanban (so it proves that we respect our scout rule…). This tool is also useful to identify which tickets didn’t respect the scout rule: we look at it during our retrospective, identify them and organize a specific retrospective to understand why the ticket took longer and take actions to prevent the same case in the future.
It’s not a magic formula, just what it’s right for our team: we tried and it worked!
- The scout rule is less abstract for the team than story points so it’s easier for them to estimate the tickets,
- We can predict the duration of a project easier with the Product Manager and define the priorities according to that,
- We used to have leftovers every sprint and now, it’s completed as planned at the beginning of the project.
But isn’t agility that: being able to question our certainties to always improve ourselves? And this applies not only to the product we build, but also to our working methods.
PS: This article is not a JIRA advertising, it’s simply the tool we are using, this method can work with any relevant tool 😉
PS 2: if you are wondering why I used the word “pace” a lot, well it’s a nod to my wonderful team at Doctolib which is called… PACE 💙 (don’t try to find the meaning of this acronym, even us, we’re having trouble remembering it…).
