In-Depth: The Double-Edged Sword Of Diversity In Teams
Results from a scientific study we performed on diversity, conflict, and psychological safety in Agile teams.
Results from a scientific investigation
How diverse is your team? Do you have members from different age groups, different genders, different roles, and different cultural backgrounds? Is that diversity beneficial to team performance, or is it challenging, or both?
This was the research question of an academic study I performed with Daniel Russo, Ph.D. The paper has been peer-reviewed and published in the journal “Transactions on Software Engineering”. This post is a non-technical summary of our investigation.
I want to thank the 500+ patrons of The Liberators for supporting this research. Their financial support makes it possible for us to engage in serious scientific research with strong practical implications. This study alone took over 400 hours. We feel it's worth the effort because we want to elevate the level of discussion in our community away from opinions and more toward evidence. If you believe this is important, and want to support our research, you can support us at https://www.patreon.com/liberators.
We generally define diversity in teams as heterogeneity in member attributes such as age, gender, cultural background, role, or personality traits (Phillips & O’Reilly, 1998). Many scientific studies have attempted to learn how teams benefit from heterogeneity. As we will discover, the answer is far from clear-cut. By reading this post, you will learn different perspectives on diversity in teams. You will learn how different types of diversity have different effects on teams. And we will offer practical recommendations based on the evidence at hand.
The Evidence To Date
In this section, we explore the scientific view on diversity to date. I will begin with the data from empirical studies, and then cover models that explain the mixed results.
Many studies have investigated the link between diversity and team performance. Their results are mixed. Tshetshema & Chan (2019) reviewed 35 studies of diversity and concluded that a negative effect is the most reported result. Similarly, Patrıcio & Franco (2022) summarize from 80 studies that diversity has a positive influence on performance because it brings more perspective, but at the cost of increased conflict due to misunderstanding. Bowers, Pharmer & Salas (2000) found a negative effect of diversity for tasks of low complexity, but a positive effect for tasks of high complexity. A meta-analysis of 30 empirical studies by Horwitz & Horwitz (2007) found no significant effects for age, gender, and cultural background, but they did find a positive effect of role diversity.
For Agile teams specifically, Lee & Xia (2010) studied 399 Agile projects and found that diversity increases quality, but not speed, at the cost of more conflict. Melo et. al. (2013) performed a multiple case study of Agile teams in three Brazilian companies. They found that diversity in experience made teams more productive. Russo & Stol (2022) surveyed ~500 engineers and found that people of different genders bring different perspectives, which is beneficial for teamwork. Rodríguez-Pérez, Nadri & Nagappan (2021) came to a similar conclusion after reviewing the literature on diversity in Agile teams. Compared to broader team research, investigations of diversity in Agile teams have focused primarily on gender diversity.
What should we make of such mixed results? Clearly, the link between diversity and performance depends on other variables. Several studies have already found the effects of team climate (Bodla, Tang & Tian, 2016), task complexity (Bowers, Pharmer & Salas, 2000), management support (Wickramasinghe & Nandula, 2015), and even time (Steffens, Terjesen & Davidsson, 2012). We will now turn to models that explain why diversity sometimes benefits team performance and sometimes harms it.
Theories For Diversity
One way to understand the conflict is to look at theories that attempt to explain those results. Two primary theories have been proposed to date to explain the evidence:
- Cognitive resource diversity theory derives from cognitive psychology. It treats teams as information processors where members process information and then integrate it as a team (Hinsz, Tindale & Vollrath, 1997). Diversity allows teams to bring varied cognitive resources — backgrounds, perspectives, norms, viewpoints — to bear when information is processed, which allows a richer exploration of shared challenges.
- The similarity-attraction paradigm (Phillips & O’Reilly, 1998) derives from social psychology and social categorization to argue that similarity between members increases mutual attraction, integration, and communication, which in turn improves performance. Diversity of members, on the other hand, results in more conflict and misunderstandings as people categorize themselves into different subgroups.
It is clear that both theories offer different predictions in the face of diversity. The first expects positive results, whereas the second expects increased conflict and (as a result) lower performance. This is the double-edged sword of diversity. Since the evidence to date supports both effects, scientists are now looking for ways to integrate these theories into one.
The most successful one to date is the categorization-elaboration model (CEM) by Van Knippenberg, De Dreu & Homan (2004). It has received broad empirical support (i.e. Guillaume et. al., 2017, Kearney, Gebert & Voelpel, 2009, Homan et. al., 2007). The CEM identifies a number of contextual variables that determine if diversity is beneficial or not, including task difficulty, trust, commitment, and task interdependence. For example, if teams work on tasks where members don’t need each other to complete their work (low interdependence), there is less opportunity to benefit from diversity. The CEM also explains that social categorization in diverse teams can activate harmful stereotypes unless there is sufficient trust and open communication. For example, when a woman enters into a team of men, the men may make negative remarks or derogatory jokes about her based on her gender. This leads to identity threat; people feel threatened because of their identity, or because others in the team do not match theirs. Consequently, the team fractures along diversity lines as conflict and tension increase and members withdraw from teamwork.
The strength of the CEM is that it explains both the potential positive and negative effects of diversity. It also shows that factors related to the team (trust, commitment) and the task (complexity, interdependence) shape how the effects of diversity work out. The CEM is more expansive than I describe here, and I refer interested readers to Van Knippenberg, De Dreu & Homan (2004) for a much more thorough overview.
I’ve now provided a brief overview of the evidence for how diversity is linked to team performance, and a variety of theoretical models that explain the link. This brings us to the study I performed with Daniel Russo, Ph.D..
Our Study
The aim of our study was to investigate how diversity in gender, role, cultural background, and age impacts the effectiveness of Agile teams. As I mentioned before, studies of Agile teams have mostly focused on gender to date. We expect a positive effect of all types of diversity on team performance. The model we’re testing for this study is shown below:
Building on parts of the categorization-elaboration model, we also want to test if psychological safety allows teams to more effectively leverage the benefits of diversity. We expect that diverse teams with high psychological safety will experience less conflict and be more effective than diverse teams with low psychological safety.
To test our hypotheses, we collected data from 1.118 team members of 161 teams. With a questionnaire we deployed through the Scrum Team Survey, we collected information about the level of psychological safety in the team, the presence of relational conflict, and overall team effectiveness. We also asked members to pick their age group, role, and cultural background. Gender was measured on the team level (estimate of % of men, women, and others) because of privacy concerns.
We analyzed the data with Structural Equation Modelling (SEM). This is an advanced statistical technique that allows us to test the full model and all expected effects in a single run. Our model fitted the data well (Chi2(129) = 156.282; TLI = .981; CFI = .988; RMSEA = .036; SRMR = .051). This means that we can use our model to test our hypotheses. Furthermore, gender diversity and psychological safety explained 40.7% of the variance in team effectiveness. For studies in the social sciences, values above 26% are considered “large”. So our model is a good starting point to test our expectations.
So how did the results match our expectations?
- Only age diversity positively contributed to team effectiveness (H1b, beta = .213, p < .05). We did not observe the effects of diversity in gender, role, and cultural background.
- Only gender diversity increased relational conflict (H2a, beta = .161, p < .05). We did not observe the effects of diversity in role, age, or cultural background.
- We did not observe an effect of relational conflict on team effectiveness (H3). We expected a negative effect.
- We did observe a very strong positive effect of psychological safety on team effectiveness (H4, beta = .660, p < .01). It also greatly reduced relational conflict (H5, beta = -.636, p < .01).
- Psychological safety, or the lack thereof, did not moderate the link between diversity and team effectiveness (H6). We expected that teams with higher safety would be better able to benefit from diversity. Our data does not support such an effect.
The statistical results are shown below:
Discussion
It is unclear why the other types of diversity (gender, cultural background, and role) did not meaningfully impact team effectiveness. It is also unclear why psychological safety did not meaningfully change the ability of teams to leverage their diversity effectively. We will now explore some alternative explanations.
It is possible that psychological safety is only relevant in teams where members actively collaborate on tasks (i.e. high interdependence). It is also possible that the effect of diversity isn’t linear where each increase in diversity means higher team effectiveness. Several studies suggest a curvilinear effect, where both too little and too much diversity is harmful to team effectiveness. However, a test for such effects in our data was not significant, so this does not appear to be the case. Finally, it is possible that teams vary in the degree to which they perceive diversity as beneficial or harmful. This is called the “Diversity Mindset” by Van Knippenberg & Van Ginkel (2021). When teams see the value and benefit of diversity, they may be much more likely to leverage their diversity effectively. On the other hand, teams that see diversity as a threat or don’t recognize it all, may not be able to capitalize on it. It would be interesting to control our results for the presence of such a diversity mindset in teams.
Our results suggest that it isn’t enough to focus on team composition alone and optimize it for diversity. One assumption of diversity is that it brings different viewpoints to a team. If teams are unable to discuss these and integrate them, they will just end up with more conflict. Teams also may not recognize their diversity and thus fail to leverage it. So teams have to learn, be supported, and be coached to benefit from their diversity and to deal with the inevitable clash of viewpoints that may happen because of it. Diversity in teams is a double-edged sword if it isn’t handled properly and without attention.
Practical implications
- Teams with members of different age groups will likely benefit from the broader range of tenure and work/life experience. The benefits of other types of diversity appear more conditional on moderating factors. Organizations can assess the extent to which teams are diverse. However, psychological safety, communication skills, and diversity
mindset seem to be important moderators that organizations need to provide and encourage teams to leverage it. - When teams grow more diverse, members’ different perspectives may lead to more conflict and friction. This appears particularly relevant to gender diversity. Such negative consequences of diversity may be counteracted when teams learn to see their diversity as a strength and recognize that different perspectives can be reconciled through open dialogue and elaboration.
- Teams that operate in environments where members can openly and safely elaborate information are more effective than other teams, regardless of their diversity. They also experience much less relational conflict. Organizations do well in developing the skills, support structures, and management styles that foster psychological safety in and around teams
- Psychological safety is paramount, but it does not appear to strengthen the cognitive benefits of team diversity, nor does not it appear to buffer against negative consequences.
Conclusions
In this post, I shared the results of a scientific investigation I performed with Daniel Russo, Ph.D.. The paper is currently undergoing peer review. A prepublication is available on Arxiv for readers interested in more details.
Our investigation of data from 1.118 team members of 161 teams adds to the nuanced and complex findings to date. Diversity in gender, role, and cultural background is not clearly beneficial to team effectiveness without consideration of other factors. We did find a positive effect of age diversity. Diversity may also increase conflict in teams, as we found for gender diversity. On the whole, our findings support the notion that teams have to learn how to effectively use their diversity through coaching, training, and support structures.
Regardless of what scientific investigations eventually conclude about diversity in teams, there is also an important moral, ethical, and societal imperative. Even if diversity in teams may be challenging at times, it is important that our teams reflect the increasing diversity in our society and create more equal opportunities.
This study took over 400 hours to perform. If you think this kind of research is valuable, and you think we should do more of it, you can support us on Patreon. Find more evidence-based posts here.
We thank all the authors of the referenced papers and studies for their work.
Research Roadmap
This is the research roadmap I’m following with Daniel Russo, Ph.D.. It may change as we discover more.
- A Theory of Scrum Team Effectiveness: Peer-reviewed & Published in Transactions of Software Engineering and Methodology (TOSEM).
- The Double-Edged Sword of Diversity: Peer-reviewed & Published in Transactions on Software Engineering (TSE)
- The Effects of Different Agile Scaling Approaches on Team Effectiveness and Stakeholder Satisfaction: In the final stage of peer review at the Journal of Empirical Software Engineering (EMSE).
- How Teaming Strategies Impact Team Effectiveness: Data collection is running. We expect this to be ready for prepublication in Q3-Q4 of 2024.
- The Impact of Working from Home on Team Effectiveness: Data collection is running. We expect this in Q4 of 2024.
