Effort Estimating: Person-Days or Story-Points?

Over the years, I’ve had the opportunity to work with many different software development teams that utilised a multitude of methodologies, working styles and tools. Although there were many differences between those teams, one thing that was common across all of them was estimating effort for the work they had to complete.

Before the age of Agile ways of working (even though I prefer the explanation that agile is a way of thinking and behaving), the Waterfall approach had been working out of a monolithic project plan that defined the timeline to complete the entire project in terms of activity-based phases; Analysis, Development, Testing, Training and Deployment. This utilised a time-based method to figure out “how long” each task would take to complete in terms of the number of days of work for a single person, thus the term Person-Days. The project manager then decided how many people to be allocated to the task, with the aim being to split the estimated number of days equally amongst the team to meet the desired delivery target.

With the emergence of the Agile ways, we have been taught to use a new method, called Story-Points, to estimate our work productivity. Even though the Agile Manifesto was first written nearly two decades ago, and given the opportunity we have had to adapt to new methodologies and frameworks during that period, we still seem not to have grasped the concept of story-points correctly. I have seen not just team members, but also program managers and product owners (who are sometimes assigned the task of driving agile transformations), get confused or even doubt how story-points could provide a more realistic estimate over person-days leading to more predictable delivery timelines.

At the same time, it is not surprising to see how some of us still struggling to make sense of story-points. We have been using time-based methods like person-days or person-hours for so long, it is challenging to detach the effort required to complete a task from time and somehow attache to a number that doesn’t mean anything by itself.

This article is an attempt to clear the confusion around this particular aspect and highlight the benefits of using a method that supports us to collaborate as a team, deliver regularly, reflect on our current ways of working and continuously improve.

Each section of the article can be read independently of the others, so feel free to skip to whichever topic is most relevant to you.

1. What is Story-Point Estimation?

Before we understand what story-point estimation is, we have to understand what a Story is. A story is a piece of work your team is assigned to complete, which is expected to be I.N.V.E.S.T.

• Independent — can be completed without having to wait for other stories to complete,
• Negotiable — the team should be able to negotiate the work involved in the story with stakeholders,
• Valuable — must deliver value to the organisation or the customer when the story is completed,
• Estimable — should have the requirements (functional and non-functional) clearly articulated for the team to have an idea about the work involved,
• Small — can fit into the team’s delivery cadence (ideally could be completed within half of the overall delivery cadence) and,
• Testable — can be tested to confirm if it has met all the requirements defined.

Now that we have an overall idea of what a story is, let’s focus on the Story-Point estimation.

Put it simply story point estimation is an abstract value assigned to represent the effort required to complete a story from the beginning till the end by considering its complexity and familiarity.

Usually, the team takes a closer look at the requirement details provided in the story (what is required?), the acceptance criteria (how should it work?) and possible solution options available (what work is required?), to deliver the expected customer outcome. Then they pick a value from a scale of values to represent the size of the story. These values don’t provide any meaning in themselves but when two different stories are compared, they indicate if the two stories would require similar or different (less or more) effort to complete.

There are two scales teams may commonly use to asses the size of a story: T-Shirt Sizes and Fibonacci Scale.

• T-Shirt Sizes — the team adopts a size guideline that represents different sizes of T-shirts. The size list is very similar to what you’d expect to see on an online shopping website while browsing garments. Typically the range may vary from small (S), medium (M), large (L) to very small (VS), small (S), medium (M), medium-large (ML), large (L), extra-large (XL) depending on the variations of tasks they have been presented with.
• Fibonacci Scale — this consists of a series of numbers that are the summation of the two previous numbers starting with 0 and 1. This gives a series of numbers that looks like the following… [0,1]> 1, [1,1]> 2, [1,2]> 3, [2,3]> 5, [3,5]> 8, [5,8]> 11, … and so on and so forth.

It is always a better practice¹ to stick to a limited number of values, mostly three or four. This forces the team to negotiate the scope of the story with their stakeholders and split them into multiple smaller stories if the scope is estimated to be bigger than the biggest story size the team is able to accommodate.

2. How does it work?

Step 1: Identify the “Base-line” story

The team identifies a story that they are most comfortable and confident they can deliver in their current product backlog.

This story doesn’t necessarily have to be delivered in the immediate future or within the next couple of sprints; rather the team needs to be comfortable with the level of details captured in the story with very little or no assumptions made.

Step 2: Agree on the Story-Point value to be assigned

The team discusses the complexity of the requirement and familiarity with the proposed solution options before agreeing on the story-point value, T-shirt or Fibonacci. Consider playing a game of Agile Poker in which team members first make up their minds on the story-point value independently, then disclose their decision to the rest of the group at the same time, on the count of three. This prevents team members being influenced by each other’s decision (especially if the team consist of junior members or senior members who’ve joined the team recently) and allows them to identify differences of opinion within the team. The team members then discuss the differences and arrive at an agreement on the value they’d like to finally assign.

It’s better practice¹ to time-box this activity to a couple of minutes. Spending too much time discussing the story in detail will most likely diverge opinions further. Always assess story points based on the feeling you get on the complexity and the familiarity of the story, not on fine details of the requirement or the implementation.

Step 3: Size other stories by comparing them with the “base-line” story

Now that the team has defined their base-line story, they can start assigning points to stories they need to pick up as part of the next couple of sprints. The way to do this is to compare each story to the base-line story and assess if it needs less, more or a similar level of effort. They will probably need to consider factors such as:

• how clear the user requirements are
• the scope of changes; does it need to modify front-end, middle layers and back-end
• how much of the code to be modified is shared
• how much does the team know about the changes they need to complete; the domain knowledge team has on apps they need to modify
• how familiar is the team with the technology stack to be used in delivering the solution
• testing requirements defined in the acceptance criteria: functional and non-functional
• what are the compliance and regulations requirements this story needs to comply with

The team then continues to play agile poker, similar to how they agreed on the story-point value for the base-line story.

Sometimes, the team might decide to conduct a “Spike” to assess a couple of solution options before they arrive at a story point value for the actual story. Spikes are time-boxed and usually assigned the smallest story-point value utilised by the team.

If the team feels that the story-points assigned to the base-line story is not at the correct place of the scale, making it difficult for them to compare subsequent stories with the base-line, feel free to adjust the value assigned to the base-line story so it provides sufficient room on both sides for other stories to be compared (and sized) appropriately.

Over time, measurements like ‘Velocity’² and ‘Cycle Time’³ will provide the context needed by the team to measure their actual rate of delivery.

3. What Are the Benefits of Story-Points?

There are many benefits of using story-points over person-days. To make the comparison more effective, let’s consider an instance where a construction team has been asked to estimate the required effort to build a house as an analogy for completing a user story in your team back-log.

Person-Days Approach

Program Manager: “Guys, we’ve got a contract finalised for a new house. It’s a 350 square-meter double story house. Garry, you are our expert concreter, Kai you are good with roofing, Banu you know plumbing, and can someone talk to Joe about electrical stuff? Jessie, can you look after the bricklaying? And can all of you guys get back to me with an effort estimate to get this job done? You’ll need to talk to Nam in the projects team for all the relevant measurements.”

In this approach, each team member assigned to their area of expertise will most probably work independently and get back to the program manager with their person-days estimates. The program manager will collate the figures; add some extra time (most likely guessing) to support administrative and a couple of unplanned activities, before arriving at an expected completion date.

Story-Point Approach

Program Manager: “Guys, we’ve got a contract finalised for a new house. It’s a 350 square-meter double story house with three bedrooms, two bathrooms, a garage to fit two vehicles, a concrete tiled roof, brick veneer walls, concrete basement, ducted heating and cooling. How many story points should we allocate?”

Team members: “What suburb do we need to build?”, “When do we need to finish it by?” “Are we getting materials from our usual suppliers or new suppliers?”

Program Manager: “It will be in Pascoe Vale. We aim to have it completed by the end of next financial year, just before winter kicks in. We’re using the usual suppliers, except for the bathroom and kitchen stuff. We need to use a new supplier in that suburb, our usual guy doesn’t deliver there…

OK, are you guys ready to size? 3… 2… 1… Hm… you all have gone with M except Garry who’s gone with L, why is that Garry?”

Garry: “We built a very similar house in Glenroy just last month. We sized it M. However, since we are going to use a new supplier and the soil type in Pascoe Vale is P, I expect this to be a bit trickier than that.”

Program Manager: “You have a point, mate … What do you guys think?”

Team members: “Yeah, that makes sense, we didn’t realise that the soil in Pascoe Vale is more difficult to build on… sure, let’s go with L…”

[1] It’s relative — With the story-point sizing approach, the team was more interested in the high-level characteristics of the new house and external aspects that had the potential to impact the construction process, before proceeding to compare the new house to something they had built in the past.

Story-point sizing allows teams to create a unique estimating scale that suits the product they build, the technology they use and the work style they follow. It also reduces (if not removes) the risk of coming up with an estimate that can only be met by the member/s who provided the estimates or with a particular level of competency and experience.

It is extremely important to acknowledge that the T-shirt or Fibonacci values that one team have interpreted may be completely different from those of another team.

[2] It’s simple — This “feel like” approach allows the team to pack the holistic picture into a single value without having to spend time rigorously going through the individual tasks required to complete the house. This shift in the mindset significantly relieves the team from having to pay attention to rather unimportant tasks, like the number of screws they need to tighten on the roof, allowing them to focus on more critical factors like the impact of a supplier not delivering materials on time.

Person-days >> “The team needs to figure out the actual time we need to deliver this requirement right now! They need to be able to identify every challenge, risk and surprise they would possibly come across, and estimate effort accordingly and upfront!”

Story-points >> “We acknowledge there can be unknowns, unplanned work and surprises along the way. Therefore let’s keep it simple and assign a value before identifying actual delivery timelines over time. We shall identify what helped and prevented us from going fast and learn from them to size our stories better in the future”.

[3] It’s faster — The need to consider each activity involved to complete the house would be extremely time-consuming. Comparing the house to be built with one or many similar houses they have built in the past makes the entire estimating process much quicker.

Even if it was a slow process at the start, teams usually get faster over time, as their understanding of what is involved increases through each delivery iteration.

[4] Encourages team-wide understanding and agreement — With a more subjective approach to estimating the effort, story-points allows team members of different practices (engineers, plumbers, electricians and bricklayers in the analogy we used) to asses “what does it feel like” compared to their colleagues. When team members come up with a myriad of story point values, it prompts them to seek clarifications from each other. Over time, this could widen narrow views that may have been formulated by particular team members, making them realise what is involved in delivering an outcome from many other viewpoints.

It increases the chances of identifying aspects of the requirement some team members might have overlooked, while other team members clarify the reasons for providing a different story-point value.

[5] More accurate — The Scrum Master⁵ will measure the ‘velocity’² and the ‘cycle-time’³ to identify how the team is tracking in delivering stories. Over time, this will formulate more accurate answers to questions stakeholders are interested in knowing, such as…

• how many different sizes of stories can your team accommodate in the next delivery cycle/sprint?
• when is your team likely to deliver a particular feature?
• how long does your team need to complete a requirement of each size (1, 2, 3, 5… or S, M, L, XL…)?
• how many different varieties of requirements have your team managed to complete in the last delivery cycle/sprint?

[6] Covers the end-to-end delivery of a feature — person-days based estimating often considers only the mainstream activities (like development and testing) leaving behind a bunch of other tasks (such as building test coverage, updating build plans, environment configuration and deploying to multiple environments) that are actually required to complete the task. This has been the number one reason tasks estimated based on person-days overrun their allocated delivery timelines and budgets.

What’s even more challenging is, even when these factors are considered, it is extremely difficult to identify the time required for each of those tasks upfront as “something” will most probably be different with each execution cycle. For example, delays due to third-party vendors, unplanned system outages and production issues that distracted the team from focusing on the feature they were required to complete can vary with each project.

Story-points provide a number without paying attention to particular tasks and encapsulate every activity required from kick-off through to the delivery of a story.

[7] It’s transparent — Every sprint your team completes and measures will provide statistics for anyone involved to analyse, probe its performance and identify where things can be improved.

If the team has short-delivered, they have the opportunity to reduce the number of stories (thus fewer story-points) they commit in the next sprint. If they have over-delivered, they could then commit to more stories (thus more story-points). Either way, the team can identify the effects of their effort estimates continuously and effortlessly.

In contrast, using the person-days method the team often has no visibility as to how they are progressing against their initial estimates until closer to the estimated completion of the project.

[8] It’s flexible — As there is no direct correlation between the story-size and the actual time spent, teams don’t need to adjust the story-point value as a way to “correct” once more details come to light or assumptions are clarified. Instead, the team can simply continue to measure ‘velocity’² and the ‘cycle-time’³ as real-life performance indicators of their delivery.

The only effect on the team would be to realise that they now need 3.5 days to complete a size M story that used to take only 3 days in the past.

[9] It’s fun — Story-points method makes effort estimation a fun activity for the entire team, especially when a game of Agile-Poker is involved. The team is ever excited to find out what they thought against their team members. It naturally becomes a team-building activity providing silent team members with an opportunity to voice their opinions and break communication barriers.

4. What makes story-point estimating successful?

Psychological safety — Team members need to feel safe to express their viewpoints and opinions during the estimating process. An open and accepting environment will encourage all team members to be authentic with their story points and to effectively articulate the reasons behind them. If they are rebuked, intimidated or ignored, the entire team might miss out on broader aspects that could have contributed to sizing a story more realistically and accurately.

Measuring Performance — This is one of the most effective ways to motivate a team to get better at what they do. Without measuring our progress, we would be blind to what could be improved. Measuring stats like Velocity², Cycle-Time³ and Throughput⁴ will allow us to know where the team is at, and start the conversations on what needs to change for them to perform better.

It is extremely important to stress that performance measurements are not used to compare different teams. They should only be used to identify how the individual team can be better at what they do. Story-points are relative and so are all the statistics that will be measured with it.

Continuous improvement — This is tied with most of the benefits mentioned under Section 3. As the effectiveness of story-points depends heavily on measuring what came after the estimating process, it is extremely important to reflect on different story-point values the team assigned and the factors that assisted/prevented them from delivering the story optimally. Identifying what could be changed for the team to perform better will not only enable more realistic story sizing but also increase the quality of the solution delivered at completion.

Periodic reflections (known as Retros) at the end of each delivery cycle/sprint are the best way to do this as it provides the opportunity to look back while the memories are still fresh.

Cross-functional teams — As much as possible, the team needs to consist of individuals with all the different skills they need to deliver the story by themselves. This enables the team to be in control of the story-size they estimated and identify any adjustments they need to make in the future.

Empowered / autonomous teams — Providing the ability to operate without having to seek approval on every activity the team needs to perform is another key success factor to make story-sizing more effective and reliable. The idea is to cut down on the waiting time the team has to go through while they work on the delivery. The ability to make most of the day-to-day decisions by themselves not only allows the team to be more realistic with their story sizes but also allows them to take ownership of their actions and learn from possible mistakes.

Servant leadership — What this means is a person to lead the team by example, continuously exploring ways to help the team to be their best. It also involves motivating the team to take risks and try new/different methods and ideas by providing an environment where failure is accepted as a possible outcome and an opportunity to reflect and learn without retribution. This way of leadership is one of the winning ingredients for effective adaptation and of high-performing agile teams in general.

5. What’s the verdict?

Accurate effort estimating is pivotal to the success of any project; from research to manufacturing, education to healthcare. In today’s world software applications play an integral part across all of those projects.

Person-days tend to provide a false sense of accuracy and certainty at the start, disappointing everyone involved in the delivery team when they fail to meet the deadline. Story-points are an empirical measurement that provides greater transparency and predictability to teams, empowering them to be certain on what they are going to deliver by closely examining what they have been delivering in the past.

Below quote by Albert Einstien is a great way to sum it all up into two sentences.

“Learn from yesterday, live for today, hope for tomorrow. The important thing is not to stop questioning.”

Thank you for reading!

If you would like to learn more head on to Momenton and hit us up for a chat.

Glossary

¹Better Practice — See my post that explains the reason why I prefer not to use the term Best Practice.

²Velocity — The rate of progress of a Scrum Team is called velocity. It is expressed by the number of story points completed per iteration. An important rule for calculating the velocity is that only stories that are completed at the end of the iteration are counted.

Further reading: https://www.scruminc.com/velocity/

³Cycle-time — A measure of the elapsed time when work starts on an item (story, task, bug, etc.) until it’s ready for delivery. Cycle time tells how long (in calendar time) it takes to complete a task

Further reading: https://stefanroock.wordpress.com/2010/03/02/kanban-definition-of-lead-time-and-cycle-time/

⁴Throughput — Is a count of the number of stories completed in a given period (such as a week, or a sprint). It is a measure of output — how much is getting done.

Further reading: https://www.mazzlo.co/blog/agile-metrics-throughput

⁵Scrum Master — As described in the Scrum Guide, the Scrum Master is responsible for promoting and supporting Scrum. Scrum Masters do this by helping everyone understand Scrum theory, practices, rules, and values.

The Scrum Master is a servant-leader for the Scrum Team. The Scrum Master helps those outside the Scrum Team understand which of their interactions with the Scrum Team are helpful and which aren’t. The Scrum Master helps everyone change these interactions to maximise the value created by the Scrum Team.

Further reading: https://www.scrum.org/resources/what-is-a-scrum-master