The problems with velocity…

Todd Brown
May 19, 2019 · 6 min read
Metrics

A recurring observation I have had over the last 15 years has been of the misappropriate use of Velocity as a measure of team productivity. The most glaring error is focusing on this number as something that should be managed and improved upon. A metric to measure productivity across changes in approach, tooling and/or team constituency is essential. The choice should be Cycle Time not Velocity. Cycle Time which measures the amount of time from when an engineer starts working on a bit of code through to its release to production — the duration from start to the delivery of value to consumers. However any single number alone is not enough. We need a small handful to assure that everything is held in proper context .

On Velocity

The core development unit in many agile implementations is called a story. The team estimates each story, giving it a number of “points” that represent the anticipated amount of work to complete that story. This includes both functional (features) and nonfunctional work (such as testing and performance tuning). Scrum (one of the more popular agile implementations) moves in time-boxed increments called sprints. Velocity is the amount of story-points we have successfully accomplished in a sprint. It is best evened out over a period of time through an average of some sort (mean or median).

It is not uncommon for immature teams to see an increase in velocity as they become comfortable with scrum, their problems domain and their teammates. Mature teams will likely see a small sprint over sprint variance in velocity, however a significant change should not occur (this is true when maturity implies an optimized process). Because of a lack of significant variance, velocity can be serve dual purpose when looking at “amount of work”. First it allows us to reflect on what has been accomplished. Secondly (and more importantly) it serves as an indicator as to what to expect in upcoming sprints.

What changes velocity

We should only anticipate small variances in velocity, but we may witness larger swings caused by these types of changes:

The last bullet is of concern, quality needs to be first class citizen of the engineering ecosystem. Its well documented that a high quality line of code costs 67% less than a low quality line of code over the lifetime of that code base (http://www.agile-doctor.com/tag/total-cost-of-ownership/). In addition high quality code, with proper automated test coverage, allows the teams to move faster in future sprints and leads to fewer escaped defects.

If we do see a change in velocity, not caused by changing the team embers or sprint duration, this is a cause for concern. Measuring and tracking velocity is a good technique to help you estimate and right size your sprint commitments. It is a useful tool as an indicator of a change in approach (such as quality), but there is little to little to no value in presenting this number outside of the team context. Further we should never compare velocity across teams, as constituency, the work represented by “a point”, sprint duration and the tasks at hand changes significantly between teams.

On Cycle Time

Cycle time starts the moment an engineer picks up a story and completes the moment it is released to production. This is a measure which is worthy of management. In this instance “management” is the tracking and discussion of how a team might reduce average cycle time. A mandate to reduce cycle time at all costs not the intended consequence. Cycle time is fairly easy to reduce:

By breaking stories into the smallest possible increments and releasing as soon as the work is complete (and signed off) the team is delivering features to its end users as fast as possible.

Impact on Velocity

Changes in cycle time should not impactful change to velocity. Velocity is the the amount of work expected to be completed in a sprint (unit of time). Cycle Time doesn’t speak to the amount of work, it speaks to how the work is accomplished and released. There may be some minor variation in velocity, particularly as teams adapt to managing cycle time (more on the impact of Cycle Time), but the velocity should normalize prior to the point cycle time minimizes. If there is a significant increase in velocity coupled with a decrease in cycle time you might be observing the effects of Goodhart’s Law (https://en.wikipedia.org/wiki/Goodhart%27s_law):

When a measure becomes a target, it ceases to be a good measure.

Avoiding the effect of Goodhart’s Law

No single metric by itself tells the story by itself — increases may be coming at the expense of tradeoffs in other places. In the case of cycle time we want to continue to deliver a high quality product — so we could easily add three metrics to measure quality:

Escaped defect rate help assure that the product your customers are using is not being impacted by poor quality. As a rate stat it combines the cumulative escaped defects per some unit of work. You could phrase it over any unit of work as long as you are consistent. Two units of work that make sense are “per sprint” or “per story” — where the latter makes more sense if releases are more frequent.

Unit test coverage, the percentage of code that is covered by automated unit tests, is seen as a predictor of quality. The higher the unit test coverage the less likely you are to be caught by a surprise defect. The caution is that 100% coverage doesn’t mean that there will be zero defects (tests can be written poorly — bad logic and.or bad assumptions).

Cyclomatic complexity is a measure of the maintainability of a code base. It is essentially the number of lines divided by the number of branches (counted by counting IF, CASE and MATCH statements). The metric has correlated positively to teams ability to iterate quickly on defects minimizing their user impact.

Impact of Cycle Time

Optimizing Cycle time requires teams to invest in automation. The investment in automation is a short term drag on feature development, but once delivered presents an opportunity for continuous deployment (the release of code to production with no manual steps). This requires a high degree oftrust through automated tests and appropriate constraints and visibility into predictive quality metrics (e.g. test coverage, cylcomatic complexity, et al).

On Monitoring

A dashboard that covered these five metrics (Cycle Time, Velocity, Escaped Defect Rate, Cyclomatic Complexity and Unit Test Coverage) would be an great start to a software metric program. It helps provide the confidence and establish the trust that the team is performing a consistent and healthy amount of quality work and placing it in the hands of their users as fast as possible.

sample report to monitor metrics

As an engineer I found software metrics programs liberating. They provided the context, to both effort and risk, that previously lacked visibility. They are able to then cross reference the teams performance with the evolving industry research. Having a confirming outside opinion and visibility into the metrics helps develop trust and a healthier team dynamic.

Beyond Cycle Time

As engineers and engineering leaders we need to become comfortable reporting on our efficiency and progress. Visibility is the start — however deriving the effect variation on different variables have on live cycle profits is probably where you want to take this…

Nerd For Tech

From Confusion to Clarification

Nerd For Tech

NFT is an Educational Media House. Our mission is to bring the invaluable knowledge and experiences of experts from all over the world to the novice. To know more about us, visit https://www.nerdfortech.org/. Don’t forget to check out Ask-NFT, a mentorship ecosystem we’ve started

Todd Brown

Written by

A 25 year software industry veteran with a passion for functional programming, architecture, mentoring / team development, xp/agile and doing the right thing.

Nerd For Tech

NFT is an Educational Media House. Our mission is to bring the invaluable knowledge and experiences of experts from all over the world to the novice. To know more about us, visit https://www.nerdfortech.org/. Don’t forget to check out Ask-NFT, a mentorship ecosystem we’ve started

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store