The Science behind Scrum (part 3): Self-organization, and how it can be improved through team composition

This post is the third in a series of posts that I will use to explore the science behind Scrum and Agile methodologies. In this post I will discuss the importance of self-organization and how to improve it through team composition. For those interested, you can also check my previous posts about Learning Organizations and Scientific Management. Follow the RSS feed to stay up to date or follow me on Twitter (@chrisverwijs).

The concept of self-organizing teams is very important in Scrum and Agile development philosophies. In this post, I want to explore what self-organization means and how it can be facilitated. As a framework, Scrum mostly prescribes that teams should be self-sufficient when it comes to achieving the Sprint Goal and performing their tasks. But as Scrum is only a framework, it does not go into a lot of detail on how to actually achieve self-organization within teams and which factors are important. In this post I will discuss some of the most important factors as identified through scientific research with teams. The post will also be sprinkled with practical tips (often in the form of the well-known bullet lists).

Why is self-organization important?

Self-organization is the process whereby a team achieves its Sprint Goal by selecting the most efficient path, given available resources, skills, abilities and knowledge. Self-organization requires (and implies) that teams actively experiment with approaches, learn from failures and continuously adjust. This is why self-organization and the concept of ‘learning organizations’ (see this post) are almost synonymous. In the context of Scrum, self-organization should probably be called ‘self-optimization’.

Why, then, is self-organization important? Scrum (and other Agile methodologies) assume the following advantages of self-organization:

• More efficient team process: As teams are responsible for achieving the Sprint Goal and are given quite a bit of freedom, teams will be more likely to select a more efficient path. This requires that teams are cross-skilled in many different ways;
• Facilitate learning: Although self-organization requires learning, learning in itself is an advantage of self-organization. As teams learn from failures and successes, they hopefully self-optimize;
• Improves ownership: When teams are given the autonomy to self-organize, they will feel more responsible for the Sprint Result. This improves the sense of ownership that a team feels;
• Improves motivation: There is a wealth of scientific literature that shows that increased autonomy (a requirement for self-organization) improves motivation within the team. This mirrors the results of socio-technical systems (see this post);
• Quality and speed: As teams learn and self-organize, they become increasingly efficient. This improves speed (velocity) and quality. This is the most difficult assumption, as many teams self-organize but don’t actually improve on objective measures (number of bugs, velocity);

Scrum states that the following rules must be implemented to achieve self-organization:

• Democratic (servant) leadership: There is no ‘project manager’, only a Scrum Master that makes sure that the rules are adhered to and removes impediments to self-organization. The Scrum Master does not micro-manage and is mostly there to remove impediments to self-organization;
• Sprint Backlog is fixed: By fixing the Sprint Backlog, the team is given freedom to implement the selected functionality within that sprint as they see fit;
• Cross-functional: Not a rule, but more a prerequisite; cross-functional teams are far more likely to self-organize. In fact, without cross-functionality self-organization is hardly possible;
• Inspect-adapt cycles: The many inspect-adapt cycles (daily scrums, retrospectives) allow teams to continuously inspect and adapt their development process;

Scrum does not go into more detail, so we’re left to figure that out for ourselves (which is ok, Scrum is just a framework). But thankfully there is a wealth of scientific literature on how to facilitate self-organization. I will discuss the most important team composition factors below. Other factors will be discussed in future blogs (like servant leadership, communication styles, conflict management, etc.).

Cross-functionality in teams

Self-organization within teams requires that members are cross-functional. This means that individual members are cross-skilled to the extent that work can be distributed dynamically at any moment in the sprint. Why is this important? Imagine a traditional team with highly specialized members; a visual designer, a tester, a DBA, a frontend coder, a backend coder and an analist. Although this team can be very efficient, it will run into bottlenecks eventually:

• Sudden unavailability: What happens if the DBA suddenly becomes ill? The other members cannot take over because they lack the skills and knowledge of what was done;
• Overburdened members: What happens when the sprint does not go well for the frontend coder? The amount of work greatly exceeded expectations and the coder cannot finish on time. The team cannot help, as other members are too specialized;
• No shared responsibility: If the analist does a bang-up job, who will feel responsible for the failure? The team will probably not. After all, the analist messed up;
• No shared knowledge: If all members work on their own specialized areas, who knows how the whole application works? The analist knows his part, and the coder his. But if they don’t actively share that knowledge, the team is at risk of losing important knowledge if one of the members leaves;

A cross-functional team will probably not run into these problems. In fact, Scrum states that all members in a team are effectively ‘developers’, thereby removing the symbolic distinction between areas of expertise. So what can we learn from this for our Scrum Teams?

• Remove roles: Avoid giving individual members a designated role other than ‘developer’. In this case, ‘developer’ simply refers to the shared responsibility of developing the sprint result. Yes, analists, designers and testers are also developers in that sense of the word. Although mostly symbolic, this breaks the mindset that someone is a ‘frontend developer’ and is therefore not responsible for the backend;
• Cross-train members: Allow teams to cross-train their skills by teaching others in the team. Pair programming is an excellent way to do this for development work. Design sessions can be used to share visual design skills. Involving all team members in meetings with the product owner will allow teams to learn analist skills;
• Address resistance: There will always be some resistance to cross-functionality. Sometimes, people are afraid to take on new roles or share their skills in fear of losing their value. Be sensitive and don’t push changes. Address any resistance openly within the team or one on one;
• Don’t overdo it: Cross-functional teams don’t mean that all members should be equally good at the tasks of others. This is impractical and quite probably impossible. But the team should be cross-functional to the extent that it has few bottlenecks in terms of team composition;
• Rotate tasks: Teams will learn most when rotating tasks. This avoids that some members always take care of testing or visual design. Rotating tasks is not always practical, especially when time pressure is high. But help teams to rotate tasks whenever possible. It might be an investment at first, but it will pay of later as time pressure can be shared within the team;

Scrum limits ‘cross-functionality’ mostly to skills and knowledge. But there are also other team factors that benefit from diversity and partial overlap. I will discuss some below.

Cognitive diversity in teams

When members have different skills, knowledge and abilities, a team is considered to be ‘cognitively’ diverse. Research has shown that cognitively diverse teams outperform homogeneous teams when it comes to gathering, processing and applying information (Justesen, 2001). Why?

• Absorbing knowledge and linking ideas: Cognitively diverse teams can more easily absorb new knowledge and generate ideas by linking previously unrelated units of knowledge on the team level (Cohen & Levinthal, 1990). This makes sense, as bringing people with diverse backgrounds and mindsets to a brainstorm will often result in more useful ideas;
• Capable of dealing with unforeseen problems: As a result, teams can more easily deal with unforeseen problems by using the knowledge and skills of individuals (Nonaka & Takeuchi, 1995);
• Many different strategies: In diverse teams, individual members will employ different cognitive strategies to solve problems. On the team level, this will translate into a larger arsenal of possible solutions. Take a Scrum Team that is faced with an almost untraceable bug in a piece of software that needs to be delivered at the end of the sprint. Different members may employ different strategies; use the debugger, write unit tests, trial-and-error, take a walk or check Google for similar bugs. Trying multiple paths will more quickly result in success;

So what can we learn from this for our Scrum Team? By promoting cognitive diversity within teams, teams can benefit from a much larger arsenal of possible strategies for dealing with problems and opportunities. But cognitive diversity is only beneficial if a team is aware of this diversity. If members have little knowledge of the skills, abilities and knowledge of others, little will be gained. The following interventions might be useful:

• Skill Awareness Sessions: Allow members of a team to describe, in their words, the skills that other members of a team possess. It might help to first try to brainstorm on possible (and relevant) skills within a team, and then linking them to individuals. This will also show which skills are lacking;
• Coaching: Coaching individual team members will help gain insight into their skills. Most people are not very aware of their skills, until they’re being asked to think about them. Invest in existing skills and new skills;
• Improve diversity: Allow members to learn skills that the team feels are missing from the team. If a team has no knowledge of formal testing, allow some members to learn a bit about Unit Testing. In my teams, this happens on a regular basis. One member might pick up a new framework in their own time and teach the others during a weekly TechTalk;
• Identify Team Skills: Identify skills that are relevant for teams within your company. Involve teams in setting up this list. Not only will these lists be incredibly useful when evaluating the skills of new and existing team members, it also helps chart the skills that exist within teams;
• Actively learn: When the team runs into failures or succeeds in achieving difficult goals, use the retrospective to specifically address the strategies that the team used to achieve the goal. Brainstorm about alternative strategies and how they might have worked. This helps the team to widen their arsenal of strategies;

Belbin and Team Role Diversity

Meredith Belbin (1981, 1993) was one of the first researchers to address the question on how team role diversity can facilitate self-organization. Although Belbin’s work mostly focused on management teams, his insights have been extended to teams in general. Belbin argued that members play different roles within teams. He identified nine key roles with his Team Role Inventory (see Wikipedia for more information):

• Plant: The ideas person who brings innovation and creativity to goals and activities;
• Resource investigator: Good at creating useful contacts outside the team. Gathers useful resources and manages interactions across team boundary;
• Coordinator: Organizes team operations and resources to meet its objectives. Good at evaluating and handling people and maximizing the team members’ potential;
• Shaper: Establishes objectives and priorities. Often has an arousing, challenging and motivating effect on others, but at times can be disruptive;
• Monitor/Evaluator: Helps team analyse and evaluate objectively and prevents impulsive decisions being made;
• Team worker: The builder of team relationships and team spirit. Helps reduce levels of interpersonal conflict;
• Implementer: Often performs tasks that others do not want. Systematic and efficient planning to transform plans into practical actions;
• Completer/Finisher: High contribution of effort and strong attention to detail. Good at planning and carrying through. Helps ensure team progresses the activities necessary to achieve goals and deadlines;
• Specialist: Provides knowledge and skills in rare supply. Only contributes on a narrow front;

According to Belbin and his collegeaus, teams with more diversity in team roles will outperform more unbalanced teams in terms of decision making, quality of the result and the team process and efficiency. Teams with only plants, for example, will struggle to survive as they lack the roles to structure the process and get the team to the desired goal. Members can play different roles in different teams. They can also play multiple roles in the same team, depending on the situation. Roles can also change over time. But the bottom-line is that diversity is needed. Belbin’s assumptions are strongly supported by empirical studies (Senior, 1997).

So what can we learn from this for our Scrum Team? When Scrum talks about cross-functionality, it is mostly concerned with knowledge and skills. Although Belbin acknowledges the relevance of skill diversity, he focusses on role diversity. As Scrum Coaches, we can use this for:

• Periodically identify the roles: For my teams, I periodically ask members to enter an abbreviated Team Role Inventory (http://www.belbin.com/) (a free, but strongly abbreviated version, can be found here: http://www.123test.com/team-roles-test/);
• Reflect and learn: Reflect on the roles with the team. Identify the roles (per member) that are most strongly present and those that are least present. Learn from these insights;
• Switch members by roles: Although I believe that team composition should remain as constant as possible, members will sometimes switch (for learning experiences or other reasons). Whenever possible, try switching based on their roles. If a team lacks a plant, try to find one;
• Selecting new members: When selecting for new members, try to find members that take on roles that are not yet present;

Personality diversity in teams

Teams can also be composed of different personalities. Some individuals are more introverted while others are more extraverted. Many studies have shown that teams that are more diverse in terms of personalities are more creative (Buchanan, 1998; Driskell et al. 1987). Most of these studies use the Big Five Personality Index by Costa & McCrae (1992) and distinguish between five ‘traits’ that all individuals possess to some degree:

• Periodically identify the roles: The degree to which someone is open to novel situations and experiences. People that score high on this trait are (intellectual) curious, adventurous and slow to judge;
• Conscientiousness: The degree to which someone is disciplined, plan and structure their work and life and are sensitive to meeting (or not meeting) expectations of others;
• Extraversion: The degree to which someone derives joy and pleasure from social encounters with others and requires stimulation to remain active. People that score high on this trait find relaxation in social events. People that score low on this trait find relaxation in being alone. I personally always like to explain the difference this way; extraverted people ‘think’ while talking to others, while introverted people first ‘think’ of something to say and then talk;
• Agreeableness: The degree to which someone is considered friendly and cooperative. People that score high on this trait seek harmony with others;
• Neuroticism: The degree to which someone experiences negative emotions, fears and anxieties and is emotionally reactive to situations. People that score high on this scale respond emotionally to situations (usually negative), which those that score low are (in that sense) more stable and respond less emotionally;

An online version of the Big Five test can be found here. Although there is continuous debate amongst researchers on the exact number of traits, a certain degree of variety in personalities within a team is beneficial to creativity. This makes sense. Teams that contain conscientious members will be more likely to meet their goals, as at least someone will feel a strong need to structure the team’s process. Extraverted members are useful in meetings with Product Owners, as introverted members tend to see which way the wind blows first. Introverted members, on the other hand, are less easily bored and might take on the role of ‘Plants’ within teams.

But wait! Are Belbin’s team roles not the same thing? Of course there is substantial overlap, but they are not the same. Personality traits are considered (by most researchers) to be more or less fixed across situation. Which team role a member plays depends more on the context (like average age, available skills, other roles in the team).

So what can we learn from this for our Scrum Team?

• Identify traits: Although I don’t think it’s useful to select teams solely on the basis of member personality, I do think it is useful to at least create awareness within the team of the personalities of its individual members. Entering an online Big Five test is a good start. Use the discussion in the team to see what can be learned from this;
• Personalities don’t really change: Accept that personalities can’t really be changed. If a member is not very agreeable or a bit neurotic at times, teach the team how to deal with this. This is especially useful when learning teams how to deal with team conflicts. Communication (soft) skills are incredible useful here. A good example is my own high score on introversion. I know that I’m not a quick thinker during meetings with the Product Owner. I have a very strong need to completely understand what someone says (an introverted trait), which makes it hard for me to think and talk at the same time. I therefore prepare a lot of questions for a meeting (upfront thinking) and take someone along who can help with the talking when I need to think about something. If I’m open about this to my team, they know that I’m not silent because I’m bored or that I’m dumb;
• Team selection: It is a good idea to at least get a general idea of someone’s personality before they join a team. If a team already consists of mostly introverted members, it might be better to add an extraverted member. This will also help in balancing out team roles;

Conclusion: Scrum Master and Team Impediments

In this post, I talked about self-organization and how it is affected by team composition. I discussed cross-functionality, cognitive diversity, team role diversity and personality diversity. I tried to lace the post with scientific research, as this hopefully grounds my post in fact rather than (my) opinion. It also shows the wealth of knowledge that is already available to Scrum Practitioners.

In future posts, I will investigate other factors important to self-organization, like communication patterns, social cognitions, conflict resolution skills, management styles and decision-making styles. But the bottom-line is that self-organization is not something that will happen automatically. Although Scrum creates a good set of boundaries within which self-organization can take place, it will require constant attention from the team and Scrum Master to keep everything progressing. Thankfully, Scrum tasks the Scrum Master with the task of removing impediments to self-organization. This post hopefully gives some insights in to some of these impediments and how to resolve them.

You can already support us with $1/month. Find out more on patreon.com/liberators

References

Belbin, R. M. (1993). Team Roles at Work, Butterworth-Heinemann, Ltd, Oxford;

Belbin, R. M. (1981). Management Teams, John Wiley & Sons, New York;

Buchanan, L. B. (1998). The impact of Big Five personality characteristics on group cohesion and creative performance. Unpublished Doctoral Dissertation. Virginia Polytechnic Institute and State University;

Cohen, W.M. & Levinthal, D.A. (1990). Absorptive capacity: A new perspective on learning and innovation. Administrative Science Quarterly, 35, 128–152.

Costa, P.T., & McCrae, R.R. (1992). NEO PI-R. Professional manual. Odessa, FL: Psychological Assessment Resources, Inc.

Driskell, J., Hogan, R., & Salas, E. (1987). Personality and Group Performance. In Hendrick, C. (Ed.). Group Processes and Intergroup Relations (pp. 91–112). Newbury Park, CA: Sage Publications;

Justesen, S.L. (2001). A study of the dynamics inherent in the relationship between innovation and diversity. Master’s thesis, Copenhagen Business School, Copenhagen, Denmark. Document retrieved on January 3rd, 2006, from http://www.innoversity.org/InnoversityPaperII_May2001.pdf

Nonaka, I. & Takeuchi, H. (1995). The knowledge creating company. New York: Oxford University Press;

Senior, B. (1997). Team Roles and Team Performance: Is there ‘Really’ a link? Journal of Occupational and Organizational Psychology, 70, pp. 241–258;

Originally published at blog.agilistic.nl on March 4, 2012.