In-Depth: Does Scrum Only Work In Small Organizations?
A surprising exploration of scientific evidence and data from real Scrum teams to learn if size matters or not
A common belief among Agile practitioners, including myself, is that Scrum works best in smaller organizations. This belief is also often expressed by people who work in large organizations themselves, and just don’t see it work.
Together with Barry Overeem, I’ve been fortunate to teach, mentor, and coach Scrum teams in many different organizations, from small to very large. I personally believe Scrum works better in smaller organizations. Things just seem much simpler there, with fewer dependencies, less red tape, and a stronger “let's get it done”-attitude. Larger organizations seem to suffer from Zombie Scrum far more often. At the same time, this is just a belief. And it's probably biased because most of my personal experience as a member of Scrum teams originates from smaller organizations.
Fortunately, this is also a belief that we can actually test against evidence. There are scientific studies that have investigated this question. And there is also a ton of data from our Scrum Team Survey that we can use. So let’s see what the evidence has to say!
This post is part of our “in-depth” series. Each post discusses scientific research that is relevant to our work with Scrum and Agile teams. We hope to contribute to more evidence-based conversations in our community and a stronger reliance on robust research over personal opinions. As you may imagine, it takes a lot of time to compile such evidence-based posts. If you think our content is valuable and we should write more of it, you can support us on Patreon.
Preface: About Our Data
In addition to the other scientific studies we cover in this post, we also run analyses on data from our own Scrum Team Survey. We created this tool for teams to diagnose and improve their process in an evidence-based way. You can use the free version for individual teams. For this post, we used a sample of 1.963 Scrum teams and 5.273 team members. The sample was cleaned of fake and careless responses and corrected for social desirability where relevant. Our sample includes teams from all sectors, sizes, and regions.
A strength of this dataset is that it represents real teams from across the globe that use it to identify real improvements. We performed a scientific study with Prof. Daniel Russo from the University of Aalborg to identify five core factors that explain a large part of what makes Scrum teams effective.
Finding #1: Scrum Is Used By Organizations Of Varying Sizes
Our dataset for this post consists of 1.963 Scrum teams from all over the globe. The first question we can answer is if Scrum is indeed more prevalent in certain sizes of organizations. The distribution is shown below:
We do see a pretty good spread across different sizes, although small organizations (1–50 employees) are less prevalent in our dataset. About a third of the teams work in average-sized organizations (51–500), and about a quarter each in large (501–5000) to very large (over 5000) organizations.
Now, our dataset may not be entirely representative. But the VersionOne report from 2021 also suggests a pretty even spread, although their buckets are different and they summarized participants and not teams. However, for the purpose of this post, we can conclude that Scrum is used in organizations of any size.
Finding #2: There Are No Meaningful Differences In Scrum Team Effectiveness Across Organizational Sizes
The second test is to see if teams in different organizational sizes score differently on the core factors of effective teams in the Scrum Team Survey that we identified in this scientific study with Prof. Daniel Russo. These factors — continuous improvement, responsiveness, stakeholder concern, team autonomy, and management support — explain much of how effective Scrum teams are at satisfying stakeholders and team members alike.
The results from an ANOVA show a mixed result. We do see significant differences in all factors (p < .05), except team autonomy. Interestingly, a trend in the result seems to be that large organizations (501–5000) seem to do a bit better on every factor. However, any observed differences are also very small, with most around 0.1 or 0.2 on a scale from 1 to 7. We can tell from the broad error bars that scores do vary a lot from team to team, but this is due to other factors.
Of particular note are the error bars for management support, which are much larger than for other factors. Apparently, the level of management support varies much more strongly than the other factors we measure in the Scrum Team Survey.
We also performed a secondary analysis by regressing organization size on the effectiveness of their teams with linear regression. This allows us to tell if the organization size meaningfully predicts (even a small part of) how effective teams are. But this regression was not significant (p=.052), which means there is no substantial effect.
So what does this mean? In a general sense, organization size alone doesn’t make a substantial difference in the core processes that make Scrum teams effective. Teams are not more or less autonomous in large organizations compared to smaller ones. Neither are they more or less responsive, more or less engaged in continuous improvement and stakeholder concern. The support that teams receive from management also isn’t meaningfully different based on size alone.
“In a general sense, organization size alone doesn’t make a substantial difference in the core processes that make Scrum teams effective.”
Thus, my belief that Scrum works better in smaller organizations is on shaky ground. But how do the stakeholders themselves feel about the outcomes they receive? Here too, I would expect better results. We will explore this next.
Finding #3: Stakeholders Are More Satisfied In Large Organizations Than In Smaller Or Even Larger Organizations
The Scrum Team Survey also allows teams to invite their stakeholders to evaluate the outcomes on four dimensions: quality, responsiveness, release frequency, and team value. We always encourage teams to invite their actual stakeholders (e.g. users and customers), and not just internal stakeholders who are only professionally involved (e.g. marketing, management).
We were able to analyze the results from 857 unique stakeholders who evaluated the outcomes of 245 Scrum teams. 49% of these represented users, 28% represented customers and 22% represented internal stakeholders. This pattern was roughly similar for stakeholders of all sizes in each organizational size. We performed another ANOVA to test if stakeholders evaluate the outcomes differently for different-sized organizations.
Now, this is where we get some really interesting results. First of all, the satisfaction of stakeholders which each of the areas is statistically different between organizational sizes (p < .05). Whereas the previous analysis yielded tiny differences, we see more pronounced differences here. Stakeholders are clearly more satisfied with the outcomes of Scrum teams in larger organizations (501–5000 employees) than in smaller or even larger organizations. At the same time, we see that stakeholder satisfaction is a lot lower overall in small organizations (1–50 employees), between 0.5 and 0.7 points. In fact, small organizations score lower than even very large organizations (more than 5000 employees), except for release frequency.
“Stakeholders are clearly more satisfied with the outcomes of Scrum teams in larger organizations (501–5000 employees) than in smaller or even larger organizations.”
This is a very surprising result. Particularly because there seems to be an optimal size (501–5000 employees). What could explain this? First, it is possible that such organizations have a better mix of resources, technologies, skills, and opportunities for teams to excel. Smaller organizations may lack such a mix, whereas even larger organizations may be more bureaucratic and constrain the use of such resources. This assumes that bureaucracy increases as organizations become larger, which is actually supported by scientific studies (Grinyer & Yasai-Ardekani, 2017). I can relate to this from my own experience. For the small companies I worked for, even The Liberators now, it was always a struggle to get funding and licenses for the proper CI/CD tooling. Since money was tight, we often had to make do with what we had. At the same time, I often found such tooling available at large organizations but restricted to particular departments or beyond complex procedures.
Second, it is possible that teams in smaller organizations more frequently deal with external stakeholders, who may be more critical than internal stakeholders (even though they are users or internal customers). Finally, it is entirely likely that larger organizations (500+ employees) more frequently resort to scaling approaches to scale work across multiple teams, which may influence their results. In any case, the effect is actually positive, particularly for large-but-not-very-large organizations. Note that we analyzed the differences between scaling frameworks (e.g. SAFe, LeSS) in a previous post, and did not find any meaningful differences.
“This is a very surprising result. Particularly because there seems to be an optimal size (501–5000 employees).”
Finding #4: Scientific Studies Show Mixed Results, But Mostly See More Challenges For Large Organizations
To our knowledge, there are no scientific studies that have specifically compared team performance and key processes by organization size. However, several studies have shed a light on how Agile methods — like Scrum — perform in organizations of different sizes, as well as common challenges. These studies allow us to understand our results better.
In contrast with our findings, several studies have pointed out that large organizations struggle more with Agile methodologies like Scrum. Mishra et. al. (2021) investigated the issues that are faced when Agile methods are embraced. They surveyed 52 software development companies. A large size, combined with bureaucracy and centralization, was identified as a negative influence on project success. Similarly, Livermore (2008) surveyed 112 software professionals and found that corporate size had a negative impact on implementation success. A limitation of these studies is that they asked participants to rate the expected impact of organizational size, rather than measuring that impact more directly. In effect, these studies could’ve simply measured a bias similar to the one I noted in the introduction. These studies also focused more on Agile transformation than ongoing performance.
Abrar et al. (2020) reviewed scientific studies to identify patterns in challenges and de-motivators in Agile adoptions based on the size of the parent organization. Few studies have investigated agile adoption in small-to-medium companies. In small companies, lack of communication was the most frequently identified challenge (100%). For medium-sized organizations, this was a lack of Agile experts (50%), continuous testing and integration (50%), and management support (22%). More studies have investigated large organizations. There, the biggest challenges are a lack of Agile experts (38%), continuous testing and integration (34%), and low management support (22%). What connects these results is that management support is clearly a big challenge. This also aligns with our finding that management support varies wildly across teams, regardless of the organization size. The fact that continuous integration and testing are commonly raised also fits. This is a particular area where a lot of skills and technologies come into play.
Dikert, Paasivaara & Lassenius (2016) inventoried 52 academic studies of the challenges that are faced by large organizations when they adopt Agile. They identified a total of 35 factors, categorized into 9 themes. The most important themes were 1) the difficulty of implementing Agile (48%), 2) integrating non-technical skills (43%), resistance to change (38%), and requirements gathering (38%). This study also emphasized many success factors, most importantly 1) management support (40%), 2) alignment of teams and management (40%), and 3) training and coaching (38%).
When we take all this together, scientific literature sees more challenges in the adoption and application of Agile methods for larger organizations than for smaller ones. This probably fits the experience of practitioners. But while this may be true for the initial implementation, our results suggest that teams in large organizations can be just as effective as teams in smaller organizations, and even better able to satisfy stakeholders. This is certainly a novel result. It should also reassure larger organizations that “size doesn’t matter”, and may even work to their benefit if they work hard to leverage the extra resources, skills, technologies, and capabilities they have over smaller organizations.
“When we take all this together, scientific literature sees more challenges in the adoption and application of Agile methods for larger organizations than for smaller ones.”
Practical Implications And Tips
So what does all this mean in practice?
- We found that the key processes that make Scrum teams effective do not meaningfully differ between organizational sizes.
- However, stakeholders appear to be most satisfied by the outcomes of teams in large organizations (501–5000 employees), and certainly more than in small organizations (1–50 employees) or very large organizations (more than 5000 employees).
- Taken together, these results suggest that large organizations are better able to provide an environment (resources, skills, technologies) to allow Scrum teams to excel at satisfying stakeholders. However, the fact that we found no meaningful differences between what Scrum teams do as part of their team processes, suggests that most of these benefits are indeed in the broad environment of teams.
- You can use the Scrum Team Survey to diagnose your Scrum or Agile team for free. We also give you tons of evidence-based feedback. The DIY Workshop: Diagnose Your Scrum Teams With The Scrum Team Survey is a great starting point.
- Our Do-It-Yourself Workshops are a great way to start improving without the need for external facilitators. The DIY Workshop: Make Learning An Ongoing Activity, Not A Scrum Event or DIY Workshop: Interview Your Stakeholders And Learn What Matters Most are great starting points. Find many more here.
- We offer a number of physical kits that are designed to make teams more effective. We have the Scrum Team Starter Kit, the Unleash Scrum In Your Organization Kit, and the Zombie Scrum First Aid Kit. Each comes with creative exercises that we developed in our work with Scrum and Agile teams.
Conclusions
My belief that Scrum works better in small organizations is clearly not supported by the evidence. The contrary seems to be true. According to our analyses, Scrum teams seem more effective at satisfying stakeholders in large-but-not-very-large organizations (501–5000 employees). This may be because such organizations have a better mix of technologies, resources, and skills to leverage Scrum teams more effectively, whereas small organizations may lack them, and larger organizations may constrain them with bureaucracy.
“So while the good news of this post is that Scrum teams can work effectively in organizations of any size, this doesn’t happen automatically.”
So while the good news of this post is that Scrum teams can work effectively in organizations of any size, this doesn’t happen automatically. But larger organizations have one clear advantage over smaller ones; their economy of scale provides them with more resources, skills, technologies, and capabilities. The challenge for them is to leverage those effectively to make Scrum teams more effective.
Good luck!
This post took 25 hours to research and write. If you think our content is valuable and we should write more of it, you can support us on Patreon. Find more evidence-based posts here.
References
Grinyer, P. H., & Yasai-Ardekani, M. (1981). Strategy, structure, size and bureaucracy. Academy of Management journal, 24(3), 471–486.
Dikert, K., Paasivaara, M., & Lassenius, C. (2016). Challenges and success factors for large-scale agile transformations: A systematic literature review. Journal of Systems and Software, 119, 87–108. doi:10.1016/j.jss.2016.06.013
Mishra, A., Abdalhamid, S., Mishra, D., & Ostrovska, S. (2021). Organizational issues in embracing Agile methods: an empirical assessment. International Journal of System Assurance Engineering and Management, 12(6), 1420–1433.
Faisal Abrar, M., Sohail, M., Ali, S., Faran Majeed, M., Ali Shah, I., Rashid, N., & Ullah, N. (2020). De‐motivators for the adoption of agile methodologies for large‐scale software development teams: An SLR from a management perspective. Journal of Software: Evolution and Process, 32(12), e2268.
Livermore, J. A. (2008). Factors that Significantly Impact the Implementation of an Agile Software Development Methodology. J. Softw., 3(4), 31–36.
