How to Evaluate Scrum Team’s Productivity?
In the early days of software development, there used to be separate teams of Development, Testing, Design, etc with a different scope of tasks to perform. I started my career as a Test Automation developer back in 2011. During my initial months, another developer and I were asked by our Test Lead to design a smoke and regression automation suite. The standard was 6 Test Cases(TC) designs per day per developer. The management decided the number based on the number of TCs and the number of people, not on the basis of other parameters like the complexity of TCs.
If the developer produces more, that becomes a standard for the team, so the expectation goes higher, and the bar also gets higher. Many times we failed to achieve this target and had to extend working hours to the weekend as well. Slowly team members learned how to tweak these numbers and they did that to escape from the unnecessary performance impact.
Being measured on how many TCs we produced the only thing we learned was how to tweak the number and reduce quality.
What was the motive for doing this?
There was pressure from management to finish test execution due to a pre-determined timeline and one way to achieve this was through increasing the productivity of teams. It means more test execution per day.
At that time, the measure of team productivity was based on the number of TCs executed per day or in line with metrics that indicate numbers that indirectly attach to the pre-determined goal of the team member.
Was it helping to improve the quality of the product? In a way, yes. If the automation suits trigger on every check-in to Master branch which provides quicker feedback. At that time, we did not have DevOps pipelines. The team used to execute in a dedicated server or system. Basically, there was the possibility of potential bug leakage to production.
From the developer’s perspective, it was one of the parameters for measuring the performance during the annual performance. So they would do whatever it takes to keep their annual review positive.
Why are we discussing all this?
We are in the era of a paradigm shift from an Industrial workplace towards Humanizing workplace. The team’s performance and productivity still have influences from the Industrial measuring techniques.
Knowledge Work like software development needs emergent solutions to solve complex problems. If we still focus on KPIs like Velocity, Throughput, Number of bugs logged, Number of Bugs fixed, etc. to measure Team’s productivity which primarily focuses on the number which could drag the team to Industrial way of doing things. Instead, our focus should be on delivering more value.
Knowledge workers are workers whose main capital is knowledge, whose job is to “think for a living”. They emphasize “non-routine” problem solving that requires a combination of convergent and divergent thinking. Will the metrics mentioned above measure their thinking ability? I do not think so. Our focus should be on the actual work produced and the evaluation criteria around those deliverables.
Is there any metric available to evaluate the team’s productivity?
Certainly, there is no single silver bullet to evaluate this. Team performance and Productivity are highly influenced by Organization Structure and Culture.
Organization Structure includes the business model, which is essentially the design for successfully operating the business. The business model includes the mission, the strategy, products, and services, and how they relate to revenue sources, a customer base, and financing. The structure also includes how employees, partners, and service providers are organized. It often influences organizational processes and policies.
Organization Culture is a body of habits that bind people together into a cohesive unit. Culture is a way of seeing things, of knowing what to do in specific circumstances. It evolves from the sum of all human behavior within an organization. It is often influenced by the organizational structure and processes, including roles, goals, and incentives.
Evolving organizational culture, processes, and possibly structure maximize the benefits of the Agile team’s performance and productivity.
Below are my favorite metrics of the productivity evaluating criteria I would prefer: Focus on the outcome, not output
1. Frequent Value Delivery:
Value could be a Services, Product, or any form of documentation to the stakeholders/customer/user that helped them to achieve certain things. Are we delivering constantly Value to customers at the end of each sprint? Are we achieving sprint goals in each sprint? I shall evaluate my team's productivity based on quality working products being delivered to stakeholders /customers rather than on some numbering metrics.
Suppose a team is constantly delivering validated value to stakeholders at end of each sprint/release with the highest quality, a good sign that team is adapting after every release/sprint. As a change leader, we need to keep in mind certain things to foster this value delivery frequency e.g.: —
- A stable team
- A self-managing team
2. Feedback implementation
This is another criteria for evaluating the team’s productivity: how fast the team implements those feedback and gets it validated.
Agile focuses on faster feedback from stakeholders/customers. Listening to stakeholders'/customers needs increases the probability of building the right product, so the entire should focus on getting the feedback faster and implementing it faster.
The above two criteria focus on the outcome which produces value to its stakeholders.
Metrics are there to measure the system, not the team or individual performance and productivity.
Scrum is not about delivering more rather delivering actual value to Stakeholders. — Scrum.org
In closing, I would say define your productivity criteria together with a Team that is understood and focus on the outcome. Kindly share your thoughts and how would you prefer to measure your Team’s productivity?
Further Reading:
Book: Drive: The Surprising Truth About What Motivates Us
Do you want to write for Serious Scrum or seriously discuss Scrum?
