Organizational Prerequisites for Scrum

I have found that when Scrum fails, it is frequently because some of the organizational prerequisites for using Scrum are not in place. This is understandable because Scrum consultants and Scrum literature usually omit to tell you that you can only run Scrum if the surrounding organization has certain characteristics. In this blog post, I will describe what those are, how you detect if you haven’t got them in place, and what you can do about it.

The following is based on years of personal experience with being a Scrum Master and Scrum Coach, not on real research.

Introduction

An agile team of software producers stands a good chance of delivering successful software. Agility can be measured as the degree of compliance to four indicators: Delivery, collaboration, reflection and improvement. Scrum is a proven process for achieving agility.

The producer team executes Scrum in an organisation consisting of themselves, the managers of the organisation and the customers that require the software solution.

My thesis in this blog post is that certain organisational prerequisites must be in place in the organisation before the team can act in a way that is compliant with the indicators. Having the prerequisites in place does not guarantee success but if any of them are absent, it is unlikely that you will achieve agility, i.e. make Scrum work.

I also give some advice on how to detect the presence or absence of the prerequisites, and how to make up for deficiencies.

Agility

First, I need to say a few words about agility.

The concept of “agile” has in many contexts lost its original meaning and become merely a fancy and noncommittal synonym for “good”. This also happened to “object-oriented” back when I was a young programmer. People even talked about object-oriented project management. Everyone and everything claim to be <insert buzzword here> because who wouldn’t want to be? And do we have a microservice architecture? Of course, we do. It’s agile too.

This is sad because these concepts have good definitions, and can be used for solving difficult problems within their areas of applicability. I recommend that you study the original Agile Manifesto to understand the philosophy behind the agile value set, and Alistair Cockburn’s Heart of Agile model to understand what agility looks like in action.

In this blog post, I will use the four imperatives of the Heart of Agile model as indicators for whether your team’s behaviour is agile or not. These are the imperatives:

• Do you deliver? That is, are you frequently building and releasing new and useful functionality?
• Do you collaborate? That is, are you communicating efficiently, sharing knowledge, status and insights? Are you helping one another?
• Do you reflect? That is, do you regularly observe and analyse the implemented solution and your process for building the solution to discover excellency, deficiencies, impediments, opportunities and problems?
• Do you improve? That is, do you regularly act on your reflections so as to make the solution and the process better?

I hope you agree that this is an eminently desirable situation to be in. Please consider your own current situation. You know whether you are doing these things or not. If your answer is a definite “yes” to all four questions, congratulations. Please tell the rest of us how you work. If the answers are more like “no”, “sometimes” or “to some degree but…”, you are quite normal and have work to do. Read on.

Organisation

Three groups of people contribute to a software solution:

• The producers are the ones actually building the solution: Programmers, testers, architects, user experience experts, data analysts and so on. Technical people. In Scrum, these are organised as a number of Scrum teams.
• The managers are the ones responsible for delivering according to contract and for managing budgets, costs, plans and people. Scrum has no real place for these people but abstracts them all away as the Product Owner role.
• The customers are the people and organisations who have an identified need for the solution and are paying for it.

Let us take the following statement for granted: If the producers execute Scrum, it is likely that they will achieve agility as defined by the four indicators in the Heart of Agile model. Corroborating this statement is important but outside the scope of this blog post.

Management and customers form an organisational environment in which the producers execute Scrum. Therefore, Scrum is only possible if the managers and customers understand, embrace and act according to the agile value set. This is true but does not tell us what behaviour is desirable and necessary on the part of the three groups. The patterns of behaviour in and between the three groups make up what I call the organisational prerequisites for Scrum. I have found that they are frequently absent. Let us look at each group in turn.

Prerequisites for Producers: Assuming Responsibility

The first prerequisite is about the producer team itself: producers must be able and willing to govern themselves, i.e. plan and act as an empowered, responsible Scrum team. This is no simple matter. Most people like the privileges that come with self-governance, but many are unable or unwilling to accept the responsibility that goes with it. Especially if it must happen on the team level, where you are not only responsible for your own affairs but the entire team’s.

On the team’s interface to the rest of the organisation, the producers must keep management informed so that they can make good decisions on the strategic level. Vice versa, management must provide clarity in the form of consistent and stable boundary conditions that the producers can understand and accept.

Also, the producers must focus on delivering value to the customers because that is the sole reason for the producers’ existence. Vice versa, the customers must make the producers understand their needs.

In my experience, the thing that is most difficult for producers to do is to assume responsibility for the solution as a whole.

An example: Many programmers are really only interested in code and technology. They don’t care about requirements or business and leave running the project to others. Programmers like this, if highly skilled, have a place in a software development organisation but if most members of a Scrum team are like this, the team cannot reflect and improve. And even if they deliver according to plan and specification, it is less likely that the solution will provide as much value to the customer as it could have.

Prerequisites for Managers: Relinquishing Control

Management must coach the producers on self-government, helping them become adept at assuming responsibility and making decisions. Also, management must facilitate improvement and change in the producer team.

Management must facilitate communication between producers and customers, ensuring that everyone is informed about decisions and that knowledge is readily available.

Empowering the producers implies relinquishing control. If a manager exercises too much control over the producers, they will stop behaving responsibly as individuals and as a team, and stop making decisions even when they see a need for change. This will result in fewer improvements of the process and the solution than should have happened, and make the producers unhappy. In my experience, relinquishing control is the most difficult thing to do for managers.

An example: A manager told me that he was willing to relinquish control to the producers but that he was concerned that they would see this as him deserting them. The remedy for this is to make it clear to the producers what the manager intends to do and why. Once the producers understand and accept this, the two parties can gradually hand over control from the manager to the producers. This must be balanced by a gradual assumption of responsibility on the producers’ part.

Prerequisites for Customers: Engaging in Software Development

The customers must continuously accept or reject work products delivered by the producers. Also, the customers must assist the producers by motivating the work that they do, by teaching them domain knowledge and by prioritising goals. An understanding of the needs that drive a solution is best conveyed through close interaction between producers and customers, not through documented requirements.

In my experience, the thing that is most difficult for customers to do is to engage in software development work.

An example: Some customers engage in discussions about the solution only at the beginning and end of a project, and are not available for advice and decision making when the details of the solution must be built and validated. This results in waste because energy is spent pondering questions that the customers easily could have given definite answers to, or on the pursuit of less important goals.

Diagnostics and remedies

Having the prerequisites in place does not guarantee success, i.e. achievement of the four indicators for agility. But if any of the prerequisites are absent, it is unlikely that you will succeed.

To help you along, here is my recipe for diagnosing lack of agility and its causes, and some advice on what to do about it. Reading the recipe is simple but executing it in your organisation will likely require a lot of work. The good news is that this can be gone gradually, and that Scrum itself provides the framework for this effort in the form of sprint retrospectives and actionable items on the sprint plan.

Start by having representatives from each of the three groups discuss and decide how well producers in your organisation are performing on the four indicators: Do they deliver? Do they collaborate amongst themselves and with customers and managers? Do they reflect? Do they improve? If the performance on any indicator is less than acceptable, move to the next step.

Next, look at the prerequisites discussed above. Find and describe any that are not in place, and how its absence causes unacceptable performance on an indicator. Also, look more generally at the relations between the three groups, and see if you can identify problems here that cause unacceptable performance on an indicator.

Finally, discuss and decide on actions. Execute and follow up. This is, of course, the difficult bit but it is exactly what (good) managers are good at managing.

Final words

Agility is desirable. Scrum can help a team become agile. But some prerequisites that are characteristics of the organisation surrounding the team must be in place. I hope that the analysis and advice in this blog post can help you realise the potential of Scrum.

By Jan Reher
Lead Systems Engineer, Systematic

About me

Jan Reher has a degree in Computer Science and has been employed at Systematic since 1998. He does programming, technical project management, Scrum mastering and coaching, teaching and facilitation. And a bit of consultancy on the side. He enjoys karate, gardening and the authorship of J. R. R. Tolkien.