Introducing the Zombie Scrum Symptom Checker
And join our worldwide research into how teams work with Scrum, and what works for them
Is your team suffering from Zombie Scrum? And if so, what can you do to improve? Do you wonder how other teams work with Scrum? How many members they have? What the usual length of a Sprint is? Is Scrum really helping teams deliver value to stakeholders faster and making them happier as a result? Find out with the first version of the free Zombie Scrum Symptoms Checker
With the first version of our free Zombie Scrum Symptoms Checker, you can now diagnose your Scrum Team. After completing a survey, you receive a profile for your team that helps you identify where you can improve. Or what you can do to recover from Zombie Scrum. When you use our site, you also help us better answer the earlier questions based on actual data. This site is an initiative by myself, Johannes Schartau and Barry Overeem and ties in with the book we’re publishing through Addison-Wesley in Q2, 2020.
In this post, I tell you more about the ‘Why’ of this checker.
The mission of our app
“Our mission with this app is to use an empirical approach to better understand how teams and organizations work with Scrum, what it makes possible for them, what enables or impedes their success and how to better support them.“
Based on a scientific approach
Scrum is grounded in empiricism. But ironically, there is little empirical research into what works and what doesn’t. The research that exists is often executed by people without extensive knowledge of Scrum. Or research remains descriptive — like VersionOne’s State of Agile Survey. Although there is nothing wrong with such research, we feel responsible for making a bigger effort to understand how teams work with Scrum and what makes them successful (or blocks them).
We are eager to use an approach that is grounded in behavioral science, with an instrument that has proven validity and reliability, and can be replicated by other researchers. Instead of purely describing what happens in the field, we want to start testing hypotheses like:
- Scrum Teams that use Sprint Goals experience higher levels of self-organization than teams that don’t;
- Scrum Teams that use Sprint Goals are better able to deliver working software every Sprint than teams that don’t;
- Scrum Teams that deliver working software every Sprint are experiencing less stress, higher morale and have happier stakeholders than teams that don’t;
In other words, we want to put the common wisdom of Scrum Masters, Product Owners, developers and people working with Scrum to the test. This helps us move from opinions to facts, and paves the way for a bright future for teams and organizations who want to work empirically with Scrum.
Developing a survey that asks the right questions and measures the right things is challenging. Wherever possible, we’ve relied on existing measures that have been validated in scientific studies. Examples of this are metrics related to team morale, team composition, team learning, and psychological safety. For other topics — like ‘inclusion of customer’s voice’ or ‘celebrating success’ — no useful measures exist. So we relied on input from experts in the field to develop initial versions. We will use the data gathered by our checker to determine their psychometric reliability and validity, and adjust accordingly. We will publish these results publicly.
Receive your team’s profile
Because entering a long survey isn’t a very appealing prospect on its own, we wanted to make it worth your while by providing you with comprehensive feedback based on your results.
So, after completing the survey for a team you receive a profile that plots your team across four dimensions: ‘Ship It Fast’, ‘Building What The Customer Needs’, ‘Self-Organization’ and ‘Continuous Improvement’. You can compare the scores of your team against the average of all teams. In the near future, you can compare against teams in your region, sector or organisational size.
In addition to dimensional scores, your team also earns badges for exceptional scores (e.g. your team is in the highest 10% of teams in terms of automation). As we learn more by gathering data from teams all over the world, we will continuously refine and improve these scores.
Optional: How our survey is constructed
Reader Tip: feel free to skip this bit if you don’t care about survey construction. It's here mostly to be transparent about our approach.
Our survey consists of a number of constructs, like ‘Involving The Customer’ and ‘Psychological Safety’. Each construct is measured with a scale of multiple questions that tap into objective aspects of each construct. For example, for ‘Involving The Customer’ one question is ‘The Product Owner actively involves stakeholders in ordering or updating the Product Backlog’.
Each construct ‘loads’ on higher-order constructs called ‘factors’, like ‘Shipping Fast’ or ‘Building What The Customer Needs’. Approaching a survey like this allows us to calculate scores for individual constructs and aggregate them into their respective factors. For example, ‘Product Vision’, ‘Including The Customer’ and ‘Managing The Product Backlog’ are currently the three factors we assume load strongest on ‘Building What The Customer Needs’. Once we have sufficient data, we can start verifying these assumptions using statistical techniques. This allows us not only to verify if our model holds up, but it also allows us to determine the validity of the scales. We will report these findings when we have them.
With statistical analyses like structural equation modeling (SEM), we can explore how constructs are related. For example, we may discover that as the size of a team grows, their cycle time drops. Or we may discover that having a strong product vision goes hand-in-hand with higher stakeholder happiness.
“For example, we may discover that as the size of a team grows, their cycle time drops. Or we may discover that having a strong product vision goes hand-in-hand with higher stakeholder happiness.”
It should be noted that we can never conclusively prove the direction of effects without experiments. We won’t be able to conclude that smaller teams result in lower cycle times. To prove that, we would need to run an experiment where we compare teams of different sizes — but similar in all other respects— against each other and measure their cycle times. But even though we can never conclusively prove causality without experiments, we do hope that showing strong correlations gives direction for further research and interventions.
“ But even though we can never conclusively prove causality without experiments, we do hope that showing strong correlations gives direction for further research and interventions.”
Open-source data and replication
Proper science is based on replication. One study can never prove anything conclusively until independent studies confirm the result. In order to encourage replication, we will do the following:
- We will share the survey, its questions, and factors as well as the psychometric properties of the scales we develop (and improve);
- We will share our anonymized dataset (with email addresses, names, and locations removed) with academic researchers who are serious about replication or further exploration and support them where needed;
- Periodically, we will publish results from our dataset up to that point or aspects we pick out;
- When time permits, we aim to publish our findings in peer-reviewed scientific journals. Or support other researchers in doing this with our data;
Next steps for our research
For us, gathering data with the survey is just the first stage. Working together with researchers, we are hoping to explore:
- The current survey is entered by one person for a team. We want to add the option to ask members of the team to fill in the survey, and then use the average scores;
- We would love to interview teams that score very high on working effectively with Scrum, to see what we can learn from them (and share with you). The same goes for teams that struggle massively with Scrum;
- We want to perform follow-up research by offering suggestions for techniques that might benefit teams with certain scoring profiles. We then want to measure how effective those techniques turned out to be according to the team, so we can build empirical grounding for what works (and what doesn’t);
- We want to identify the biggest risk factors that impede (or promote) effective Scrum or cause Zombie Scrum;
- We want to augment the measures of value from our survey with more objective, factual measures (like money saved or generated) — perhaps through interviews;
The Product Backlog for our app is publicly accessible here.
Give it a try!
We can only encourage you to give the app a try. It may only be a minimum viable product, but we are eager to add more features as data starts coming in. Go to https://survey.zombiescrum.org to try it for your team(s). And we’d love to hear your feedback, suggestions, bugs, and questions in the comments (or in our mailbox at firstname.lastname@example.org).