Psychological Safety In The Agile Community

A scientific investigation of the experience of psychological safety, the factors contributing to it, and the consequences on members — for online and face-to-face interactions.

How psychologically safe is the Agile community for its members? What does it look like when psychological safety is low? And how do members react to it?

We—Christiaan Verwijs and Evelien Roos—set out to answer these questions with help from Professor Daniel Russo. This post reflects the results of this study and provides an in-depth, detailed overview of our results and their implications.

We are in the process of preparing this study for peer review in an academic journal. This typically takes 1 to 2 years.

TLDR: Summary of Key Results

• We define the Agile community as a global knowledge-sharing community with many smaller subcommunities. Research has shown that psychological safety is an essential predictor of members’ intent to continue contributing to virtual communities.
• For in-person interactions, most participants (N=160) experience a moderate to high level of psychological safety (3.9 on a scale from 1 to 5). For online interactions, most participants experience a low to moderate level of psychological safety (2.9).
• The behaviors that participants typically experience are the rejection of (new) ideas (37.1%), tribalism (24.5%), and not feeling heard (19.6%). 4.2% reported no experience with behavior that lowered their psychological safety. The most common strategy for dealing with this is for participants to stop contributing or withdraw from the community (55.2%). A tenth of participants reported lowered emotional well-being (9.8%). 11.2% of participants report no effect on them.
• We found no significant difference in reported psychological safety by gender. However, women report substantially more dismissal of their views and tend to withdraw more from further interactions. A significant difference was found when comparing between age groups. The youngest cohort (26–36) reported a high level of psychological safety, which then dips to the lowest level (36–45) and trends up from there to the highest level for the oldest cohort (66+). We also found significant differences between roles, with facilitators and trainers reporting the lowest level of psychological safety.
• Participants identified three primary factors contributing to lower psychological safety: poor debating skills (36.4%), dogmatism (30.8%), and self-promotion (16.1%).
• According to participants, the three primary strategies to improve psychological safety are more community engagement and leadership (21%), improving the quality of dialogue (19.6%), and personal coping strategies (10.5%).
• Future studies can replicate this study to analyze trends and compare psychological safety between different communities.
• We identified several limitations to our research, which primarily impact the degree to which the findings can be generalized to the global Agile community.
• Whether the level of psychological safety is too low or good enough is a normative one that is up to the community. However, research shows that the intention to share knowledge is lower when psychological safety is low. If we combine this with observed differences in gender, role, and age, knowledge sharing may be harmed, lowering the diversity of those contributing.

Psychological Safety in (online) communities

We define the Agile community as a global community of people interested in Agile and participating to share knowledge and learn from others. Interactions can occur face-to-face at conferences, workshops, and meetups or virtually, mediated through platforms such as LinkedIn, Medium, Zoom, Substack, Discord, etc. It is not a monolithic community and is instead composed of many smaller communities that are loosely connected. This aggregation of individuals, businesses, and subgroups around a shared interest is similar to how Porter & Donthu (2008) define virtual communities.

Several studies have explored the factors contributing to knowledge sharing in virtual communities. Zhang et al., 2010 identify psychological safety as a significant predictor of the intention to continue sharing knowledge. Amy Edmondson (1999) popularized psychological safety as “a shared belief held by members of a team that the team is safe for interpersonal risk-taking”, which draws on earlier work by Edgar Schein (Edmondson & Lei, 2014). It is important to emphasize that a climate of high psychological safety is not akin to a “conflict-free zone.” Edmondson and Schein emphasize the need for healthy and constructive conflict to facilitate shared learning, though not in a way that aims to ridicule, harm or otherwise single out individuals or reduce their social standing in a group.

The experience of low psychological safety may lead people to stop their contributions to a knowledge-sharing community. Illustration by Thea Schukken.

This study aims to assess the level of psychological safety in the Agile community, both in online and in-person interactions. This distinction is important because we expect online settings to evoke different behaviors and responses. Nonverbal cues play an important role in creating and evaluating psychological safety (Kostovich, O’Rourke & Stephen, 2020; Hayashida et al., 2024; Nillson & Norström, 2024), and this is hard to replicate online. Furthermore, we want to learn what the experience of low psychological safety looks like in practice, what factors contribute to it, and how it can be improved.

Research design

To answer our research questions, we designed a multi-phasic study. The first phase aimed to assess the level of psychological safety, the factors that contributed to it, and its effect on participants. Because the study concerns the Agile community, we felt it was essential to involve that community in making sense of the results and testing the credibility of our findings. Thus, the second phase aimed to report the results and perform member checking to assess credibility and develop more insights.

Figure 1: Survey design in two phases

Both phases featured online survey studies. Such studies have drawbacks, like selection bias, but they are also convenient for quickly building larger samples. We purposefully included quantitative questions—Likert scales—and qualitative questions—open essay questions—to get numeric data and actual experiences. The survey order was also randomized by area to counter order effects.

The survey studies for both phases were advertised through various social media platforms well-used in the Agile community, particularly LinkedIn, Discord, Slack, and podcasts. We also approached well-known members of the Agile community to help us spread the word and participated in several industry podcasts to raise awareness. The survey for the second phase presented the results objectively, identified three core findings per area, and asked participants to share their feedback (see Figure 2).

Figure 2: Examples of the survey in the second phase. We presented the results, added a brief overview of results and asked participants for their thoughts.

The quantitative data from the first and second phases were analyzed with SPSS. To analyze the essay questions, we used the Gioia approach (Gioia, Corley & Hamilton, 2012) to classify and code the behaviors into 1st order concepts rigorously, 2nd order themes, and aggregate dimensions. In other words, a grounded theory describes the experiences of, the factors that influence, and the effects of low(er) psychological safety. We then counted how often participants reported the various dimensions. Each participant could provide examples from one or more dimensions.

The essay questions from the second phase were categorized using ChatGPT. Large Language Models (LLM) such as ChatGPT are ideally suited for objectively analyzing and classifying textual data (Tai et al., 2023; Chew, 2024). Such an approach is less time-consuming than the Gioia approach. Moreover, the second phase aimed not to develop a grounded theory but to identify broad textual response themes.

Sample

For the first phase, we received 160 responses. After removing careless and fake responses, our sample contained 143 participants. The distribution in our sample (by gender, age group, content creatorship, role, and seniority) is shown in Figure 3:

Figure 3: Distribution in sample by gender, age group, content creatorship, role and seniority.

Phase 2 Sample

For the second phase, a survey was distributed across the same channels as the first phase. We wanted to include both previous participants and people new to the research. We received 31 responses. One case was removed because most questions were left empty. See Figure 4 for sample composition.

Figure 4: Distribution in the sample by previous participation in phase 1.

Where do people interact with the Agile community?

This study investigates the level of psychological safety in the Agile community for online and face-to-face interactions. So, we asked participants to indicate how often and where they share something with the Agile community on social media (see Figure 5). LinkedIn is the most popular social medium in our sample, followed with some distance by forums and YouTube.

Figure 5: Percentage of how often participants share something with the Agile community by online social medium

We also asked participants to indicate where and how often they meet members of the Agile community for face-to-face interactions (see Figure 6). This included both physical interactions and video face-to-face interactions. Workshops or training are the most popular, followed by in-person and online meetups.

Figure 6: Percentage of how often participants interact with the Agile community through face-to-face interactions

We performed a power analysis with G*Power. Our sample size was large enough to detect actual differences, with a chance of 99.99% for differences between in-person and online interactions. For comparison between age groups, roles, and gender groups, our sample was large enough to detect actual differences with a chance of 99.85%.

Level of psychological safety in the Agile community

This brings us to the core questions of our study. Participants were asked to evaluate the level of psychological safety they experienced in the Agile community. We used a psychometric Likert scale (1–5) consisting of 4 modified items from Edmondson’s measure of psychological safety (1999). This scale included items such as “Members of the Agile community are open to different perspectives and opinions”, “I frequently observe or experience behavior that lowers psychological safety” and “Members of the Agile community foster an environment where is easy to ask for help”.

Each participant answered the scale twice: once for face-to-face interactions (e.g., meetups, conferences) and once for the online social medium they used most frequently (LinkedIn for most). The order was randomized between participants to counter order bias. The measurement reliability for this scale was high for face-to-face interactions (α = 0.84) and online interactions (α = .85).

Psychological safety was evaluated with a mean average of 3.88 for face-to-face interactions (N=141) and 2.91 for online interactions (N=143). The means and standard deviations are shown in Figure 7. The difference is significant (p < .001). This means that psychological safety is relatively high in face-to-face interactions and much lower for online interactions.

Figure 7: Mean averages for psychological safety in face-to-face and online interactions in the Agile community.

Phase 1: What behaviors lowered psychological safety, specifically?

The quantitative and numerical data in the previous section indicate patterns, but it does not tell us what kind of experiences people have with low psychological safety. So, we also asked participants for written examples of behaviors they experienced or observed that lowered psychological safety. This resulted in 14 pages of open text. We coded these responses into 1st order concepts, 2nd order themes, and aggregate dimensions. This resulted in the following dimensions (see also Figure 8):

• Rejection or dismissal (reported by 37.1% of participants): The dismissal of ideas, contributions, or suggestions by others. This includes ridicule, gatekeeping, patronizing, sexism, bullying and mobbing.
• Tribalism (reported by 24.5% of participants): This covers highly dogmatic interactions in which specific frameworks, approaches, or practices are bashed or discredited without evidence or clear argumentation.
• Not feeling heard (reported by 19.6% of participants): This factor reflects situations where participants don’t feel heard by others. For example, others do not try to understand (no questions, interrupting). It also includes marginalization when no room is made for discussion or debate.
• Not feeling respected (reported by 9.8% of participants): Interactions in which participants do not feel that others respect their view or person. This includes gossip, depersonalization, and other behaviors that show a lack of respect in the participants' eyes (e.g., not showing up to meetups when signed up, revisiting discussed topics, and focusing only on what is negative about a contribution).
• Harassment (reported by 7.7% of participants): This factor covers more overt behaviors that lower psychological safety. It includes personal attacks, sharing personal details to make the participant look bad, and sexual harassment.
• Other behavior (reported by 6.3% of participants): Not all the examples provided by participants fit neatly into one of the above categories. The remaining examples were grouped into an “Other” category.
• No experience (reported by 4.2%): Not all examples fit neatly into one of the categories, and this factor reflects that.
• Unethical behavior (reported by 2.1% of participants): This factor includes violations of ethical norms, violations of conference safety agreements, and when others take work done by a participant and pass it off as their own.

The most frequently experienced behavior is “Rejection or dismissal” (37.1%), followed by “Tribalism” (24.5%) and “Not feeling heard” (19.6%). 4.3% of participants did not observe or experience such behaviors at all.

Figure 8: Percentage of participants (N=143) that reported to have observed or experienced behavior that lowered psychological safety.

Phase 1: How did those behaviors affect participants, specifically?

The prior section describes what behavior participants experienced or observed that lowered their psychological safety. We also wanted to learn how this affected participants personally. So, participants were asked to write down how it affected them with an essay question. We received 143 responses, totaling 12 pages of text. We coded these responses into 1st order concepts, 2nd order themes, and aggregate dimensions. This resulted in the following dimensions (see also Figure 9):

• Withdrawal or disengagement (reported by 55.2% of participants): This covers all behaviors that indicate withdrawal or disengagement, such as exiting the community, existing social media or lowering overall engagement with others in the community through filtering, speaking up less frequently and lower motivation to post.
• No effect (reported by 11.2%): Not all participants experienced the impact of lowered psychological safety. This category reflects those responses.
• Lower emotional well-being (reported by 9.8% of participants): This factor covers emotional responses to low psychological safety. It includes anger, sadness, and annoyance, as well as emotions associated with mental health (anxiety and loneliness).
• Restrict interactions (reported by 9.1% of participants): This factor addresses attempts by participants to filter out those who lower their psychological safety, e.g., by not responding to them anymore, by avoidance, and by restricting who can comment or follow them on social media.
• Other behavior (reported by 5.6% of participants): Not all examples fit neatly into one of the categories, and this factor reflects that.
• Create safety for others (reported by 4.2% of participants): This includes behaviors and strategies to increase psychological safety for others, such as calming the waters, seeking to include others, and talking about other topics.

55.2% of participants reported withdrawing from further interactions or the community. 11.2% reported no effect on them. 9.8% reported lower emotional well-being.

Figure 9: Percentage of participants (N=143) that reported certain effects on them because of lowered psychological safety.

Phase 2: To what extent did participants recognize the results?

In the second phase, we evaluated the degree to which members of the Agile community recognized the results. This allows us to test credibility and develop further insights.

Thirty participants rated the match between their expectations and the reported psychological safety in online and face-to-face interactions. On a scale from 1 (“The results strongly differ from my expectations”) to 5 (“The results strongly match my expectations.”), The mean average was 4.17 (st. dev: .91), which indicates moderate to high recognition of the results.

Gender differences in psychological safety

We also compared various subgroups in our sample to determine differences in the experience of psychological safety. The first such comparison was by gender. Our sample consisted of 90 participants who identified as men, 43 as women, and 10 who identified otherwise or did not want to disclose this information. The group differences (mean averages) are shown in Figure 10.

Figure 10: Mean averages for psychological safety in face-to-face and online interactions in the Agile community, compared by gender. The bars represent the standard deviation.

Although men typically scored psychological safety higher than women and people who identified otherwise, gender differences were not statistically significant at p = .05.

We also analyzed differences in the qualitative experience of behaviors that participants personally encountered that lowered psychological safety (Figure 11). Women reported a substantially higher rate of “Rejection or dismissal” than men (resp. 53.5% and 32.3%). More minor differences were observed elsewhere, with women reporting more “Tribalism” (resp. 27.9% and 23.3%), and men reporting “Not feeling heard” more often (resp. 21.1% and 16.3%). 7% of women reported no experience with behavior that lowered psychological safety compared to 3.3% of men.

Figure 11: Number of participants per category that reported experience with behaviors that lowered psychological safety for them. Categories with 0 responses are not shown. The gender category “Other / prefer not to disclose” was suppressed because, for some categories, only 1 or 2 people acknowledged they experienced it, which might compromise anonymity.

Finally, we also compared the effect of low psychological safety by gender. The differences are shown in Figure 12. The most substantial difference was observed for “Withdrawal or disengagement,” which was higher for women than for men (resp. 60.5% and 52.2%).

Figure 12: Number of participants per category that reported effects of low psychological safety on them. Categories with 0 responses are not shown. The gender category “Other / prefer not to disclose” was suppressed because, for some categories, only 1 or 2 people acknowledged they experienced it, which might compromise anonymity.

Phase 2: To what extent did participants recognize the results?

To evaluate our findings, we shared our results with 30 community members for validation, see Figure 13. On a scale from 1 (“The results strongly differ from my expectations.”) to 5 (“The results strongly match my expectations.”), the mean average varied between 3.93 and 3.67, indicating moderate recognition of the results.

Figure 13: Mean averages for the extent to which results matched the expectations of participants (N=30). The bars represent the standard deviation.

We also asked participants to interpret the results for gender differences in overall psychological safety, factors contributing to low psychological safety, and effects on persons. The anonymized text responses were analyzed and classified by the LLM behind ChatGPT. It showed:

• 23–30% attributed the differences to broader societal gender biases and 17–10% to the dominance of men in the Agile community.
• 27% noted the universality of reactions to low psychological safety, regardless of gender. However, 20% also highlighted gender differences in responses.
• 13–10% of participants desired more contextual information or raised questions about the results and our interpretation. For example, one participant noted that they did not observe significant differences in the graphs.

Age differences in psychological safety

We also compared the experience of psychological safety in the Agile community by age group. Participants were categorized into 10-year buckets, starting with 26–35 as the first bucket participants. The group differences are shown in Figure 14.

Figure 14: Mean averages for psychological safety in face-to-face and online interactions in the Agile community, compared by age group. The bars represent the standard deviation.

Psychological safety is about the same across age groups for face-to-face interactions, and observed differences are not statistically significant at p = .05. However, the differences between age groups are statistically significant for online interactions (p < .001). Psychological safety starts at 3.31 for the youngest age group, then dips to the lowest level in the age group 36–45 (2.66, N=44) and trends up from there to 3.72 for the group of 66+ (N=5).

When we break down the actual behaviors experienced by age group, we observe that the youngest group (26–35) reports the highest level of “Rejection or dismissal” (50%) and “Not feeling respected” (21.4%). The age group of 36–45 reports the highest level of “Harassment” (6.8%) and “Not feeling heard” (27.3). The oldest age group (66+) reports the most “Tribalism” (42.9%). The results are shown in Figure 15.

Figure 15: Number of participants per category that reported experience with behaviors that lowered psychological safety for them. Categories with 0 responses are not shown.

Regarding the effects of low psychological safety, we also observe some differences between age groups (see Figure 16). Here, “Withdrawal or disengagement” is the most reported effect in the age group 36–45 (59.1%). Furthermore, this age group also reports the highest incidence of “Lower emotional well-being” (13.6%).

Figure 16: Number of participants per category for the effects that low psychological safety has on them. Categories with 0 responses are not shown.

Phase 2: To what extent did participants recognize the results?

To evaluate our findings, we shared our results with 30 community members for validation, see Figure 17. On a scale from 1 (“The results strongly differ from my expectations.”) to 5 (“The results strongly match my expectations.”), the mean average varied between 3.07 and 3.80. The results for the level of psychological safety by age group were least expected, whereas age differences for the factors and effects were mainly expected.

Figure 17: Mean averages for the extent to which results matched the expectations of participants (N=30). The bars represent the standard deviation.

We also asked participants to interpret the results for gender differences in overall psychological safety, factors contributing to low psychological safety, and effects on persons. The anonymized text responses were analyzed and classified with ChatGPT:

• 30% noted the universality of how people experience psychological safety, regardless of age. However, 13% also noted age differences in how people interpret and respond to situations.
• 10–22% noted curiosity about the categories of factors and effects and wanted to learn more about them.
• 22% identified a wording issue in one of the categories. “Withdrawal or disengagement” was erroneously listed as “Withdrawal or engagement,” which understandably resulted in questions about why such disparate behaviors ended together.

Role differences in psychological safety

Finally, we compared the psychological safety experienced by different roles. We distinguished between Scrum Masters (N=35), Agile Coaches (N=49), Trainers and facilitators (N=17), Consultants (N=21), and a group of other roles (N=18). The reported level of psychological safety is reported in Figure 18.

Figure 18: Mean averages for psychological safety in face-to-face and online interactions in the Agile community, compared by age group. The bars represent the standard deviation.

We observe some variation in reported psychological safety between roles for face-to-face interactions, although differences are not statistically significant at p = .05. The differences in online interactions are significant, however (p < .01). Trainers and facilitators report the lowest level of psychological safety (2.66) and the highest is reported by Scrum Masters (3.23).

When we analyze the psychological safety-reducing behaviors reported by the various respondents, we note that trainers and facilitators report the highest level of “Rejection or dismissal” (58.8), followed by Scrum Masters at some distance (42.9%). Agile coaches report the highest level of tribalism (34.7%). Scrum Masters are the only group where some members report no personal experience with behavior that reduces psychological safety (8.6%). See Figure 19 for a visualization.

Figure 19: Number of participants per category that reported experience with behaviors that lowered psychological safety for t hem. Categories with 0 responses are not shown.

Finally, we compared the effects of low psychological safety by role (see Figure 20). “Withdrawal or disengagement” is the most often reported effect by Agile Coaches (59.2%), Trainers and facilitators (58.8%), and Consultants (61.9%), and the lowest by Scrum Masters (42.9%). Scrum Masters, on the other hand, report “Lower emotional well-being” more often (20%) or report no effect at all (20%).

Figure 20: Number of participants per category for the effects that low psychological safety has on them. Categories with 0 responses are not shown.

Phase 2: To what extent did participants recognize the results?

To evaluate our findings, we shared our results with 30 community members for validation, see Figure 21. On a scale from 1 (“The results strongly differ from my expectations.”) to 5 (“The results strongly match my expectations.”), the mean average varied between 3.33 and 3.60. This indicated that the results were mostly as expected, to varying degrees.

Figure 21: Mean averages for the extent to which results matched the expectations of participants (N=30). The bars represent the standard deviation.

We also asked participants to interpret the results for gender differences in overall psychological safety, factors contributing to low psychological safety, and effects on persons. The anonymized text responses were analyzed and classified with ChatGPT

• 30% noted role-specific challenges. For example, some participants argued that Scrum Masters are expected to point out impediments but may also be seen as obstructive.
• 25% wondered if the psychological safety they experienced was related to role-specific experiences. Some roles may be more sensitive to psychological safety than others, and coaches may be more attuned to it than consultants.
• 20% reported curiosity or surprise with the differences between roles. For example, several participants wondered why trainers and facilitators report so much more rejection and dismissal.
• 50% reported confusion at the visualization we provided for the effects of psychological safety by role. We discovered that the image was indeed incorrectly replicated from a previous slide.

Factors that contribute to low psychological safety

In the previous section, we reported the level of psychological safety among practitioners in the Agile community, both in face-to-face and online interactions. We also asked participants to speculate on the causes of low psychological safety. 137 participants responded to the essay question, “In your experience, what behavior or factors decrease psychological safety in the Agile community?” This resulted in 11 pages of open-text responses.

We coded these responses into 1st order concepts, 2nd order themes, and aggregate dimensions. This resulted in the following dimensions emerged:

• Poor debating skills (reported by 36.4% of participants): The inability to articulate one’s views reasonably and logically in a discussion with others. This includes poor listening skills, not trying to understand other views, and using flawed arguments (e.g., arguments from authority, logical fallacies). It also includes aggressive language (bashing, ridiculing, personal attacks) to sidestep debating differing views.
• Dogmatism (reported by 30.8% of participants): This factor concerns rigidity in personal views and the resulting rejection of opposing views in discussions.
• Self-promotion (reported by 16.1% of participants): This factor covers all behavior that aims to promote or protect one’s services and commercial interests to the detriment of others. This includes polemics about frameworks and the creation of controversial content optimized for exposure (click hunting).
• Lack of diversity (reported by 4.2% of participants): The degree to which the setting in which interactions take place promotes diverse viewpoints or is very homogenous instead. This includes lack of minority views, lack of visibility of minority groups, lack of women, and behavior that inhibits diversity, such as racism and sexism.
• Community dynamics (reported by 2.1% of participants): The dynamics in a given Agile community and how they influence psychological safety. This includes whether or not people stand up for each other and the presence of work agreements, clear norms, and gossip.

Each participant could report one or more potential factors. Figure 22 summarizes the five core factors and their response percentages.

Figure 22: Percentage of participants that reported factors that contribute to low(ered) psychological safety

Strategies to improve psychological safety

Finally, we asked participants to suggest strategies or measures to improve psychological safety. 110 participants responded to the essay question, “What measures or strategies could improve psychological safety and encourage more respectful and constructive discussions within the Agile community?” This resulted in 10 pages of open-text responses.

We coded these responses into 1st order concepts, 2nd order themes, and aggregate dimensions. This resulted in the following dimensions emerged:

• Community engagement and leadership (reported by 21% of participants): The use of clear community agreements to set norms about what is acceptable and what is not. This factor also includes moderation (if possible) and standing up to norm violations together.
• Improve the quality of dialogue (reported by 19.6% of participants): This factor focused on educating people on debating skills and techniques, such as active listening, asking questions before sharing one’s view, using scientific evidence when available, and keeping discussions professional. Many participants also pointed at the Scrum Values or the values in the Agile manifesto as guideposts.
• Increase diversity in perspectives (reported by 7.7% of participants): Being more sensitive to diversity in designing meetups and interactions. This includes more diverse panels (with women, minorities, and outside views), explicit invites to people with other views, and being more empathic to different views.
• Personal coping strategies (reported by 10.5% of participants): This strategy covers personal actions to increase one’s sense of psychological safety. This includes unfollowing or blocking people who lower one's psychological safety, leaving social media altogether, prioritizing in-person interactions, and building resilience.

Each participant could report one or more potential factors. Figure 23 summarizes the five core factors and their response percentages.

Figure 23: Percentage of participants that reported strategies and measures to improve psychological safety

The usefulness of Phase 2 for participants

Our study consisted of one phase, where we collected data about psychological safety from the Agile community. The second phase was when we shared the results with community members to get their thoughts. Because such a two-step approach is rarely practiced, we wanted to learn how useful this was to participants. This was assessed with a 5-point Likert item (“To our knowledge, this is the first time the community actively contributes to the sense-making of scientific evidence. Did you find this process interesting?”). Figure 24 shows that the mean average varied between 4.13 and 4.38, reflecting answers between “Very interesting” and “Extremely interesting”.

Figure 24: Mean averages for expressed interest in the second phase by participants (N=30)

We also asked participants to elaborate on their scores with an open-text question. The responses were analyzed with ChatGPT, which resulted in six categories. 23% made suggestions for improvements in future studies. 20% expressed interest and engagement. 17% found the experience useful for self-reflection. 17% raised questions about certain study aspects or criticized some classifications. 15% expressed appreciation of the second phase, and 8% wished to know more about the data and the context.

Follow-up research questions generated by participants

Finally, participants were asked to suggest future research topics based on these results that were of interest to them. The responses were categorized with ChatGPT into six themes:

• 23% of participants wondered how these results applied and compared to other contexts beyond Agile.
• 20% of participants desired more practical and improvement-focused research. Which evidence-based approaches improve psychological safety, particularly in online settings?
• 17% of participants expressed interest in learning more about which factors shape psychological safety online and in face-to-face interactions.
• 15% of participants desired to explore differences and similarities between psychological safety in communities and teams.
• 15% of participants wanted to explore the distributions of scores more specifically or raised specific questions about this study.
• 10% of participants suggested more methodological approaches or asked for clarifications.

Discussion

We now turn to a discussion of the findings as reported above. The results clearly show that psychological safety is substantially lower for social media interactions than for face-to-face interactions in the Agile community. This is not surprising, as nonverbal cues play an important role in creating and evaluating psychological safety (Kostovich, O’Rourke & Stephen, 2020; Hayashida et al., 2024; Nillson & Norström, 2024), which are hard to replicate on social media.

The types of behavior that participants primarily associated with lowered psychological safety were rejection or dismissal of their views, no effort from others to understand their opinions, and tribalism. Most participants reported that they withdrew or disengaged from further interactions, or even the community altogether, as a consequence.

This brings to a question underlying this research. What level of psychological safety can be expected in online interactions compared to face-to-face interactions? The mean average for online interactions is 2.91 on a scale from 1 to 5, just below the halfway point. Let's consider the questions that were asked (i.e., “members make an effort to understand each others’ views” and “members of the Agile community foster an environment where it is easy to ask for help”). Most people don’t observe behaviors consistent with psychological safety in their interactions on social media such as LinkedIn, forums, and YouTube. Our qualitative analyses underscore that this manifests primarily in the rejection of (new) ideas, not feeling heard, and tribalism. Whether this is acceptable is not a scientific question but one the community must answer for itself. What is clear from the results is that the reported level of psychological safety causes more than half of the participants to withdraw or disengage from interactions or the community in general and that emotional well-being is lowered for a tenth of the participants. This undermines the ability of a community to facilitate knowledge sharing, as Zhang et al. (2010) also reported as a consequence of low psychological safety in online communities. A minority report no experience with low safety (4.2%) or no effect on them because of it (11.2%).

This study also investigated how psychological safety is impacted by diversity in gender, age, and role. We found no significant difference in gender in reported psychological safety. However, women reported substantially more rejection and dismissal (53.5%) than men (32.2%) and indicated withdrawal from community interactions more often than men (60.5% vs 52.2%). For age groups, our data suggests that psychological safety starts high in the youngest cohort (26–35), then dips sharply (35–46) and recovers over time. Our data does not explain why this happens. It might be an example of the “honeymoon-hangover” effect (Boswell, Boudreau & Tichy, 2005) where the initial enthusiasm for a new job dissipates as reality does not always match expectations.

Regarding role diversity, trainers and facilitators reported the lowest level of psychological safety, whereas Scrum Masters reported the highest. Surprisingly, Scrum Masters are less likely to withdraw or disengage due to lower psychological safety than the other roles. While our data does not tell us what causes these differences, it is clear that psychological safety is experienced differently by different groups within the Agile community and has different effects. However, our results also show that those who experience low psychological safety are also most likely to withdraw from (further) engagements or leave the community entirely. In the long run, this may harm the diversity of opinions and impact the learning ability of a community (Zhang et al., 2010).

Our study presents one point in time. It is unclear how psychological safety has evolved from the origin of the community in the 1990s and will evolve from here on. With the proliferation of Agile in the early 2000s, many have flocked to the Agile community in search of shared learning, curiosity, and business interests. It would be interesting to track psychological safety as it changes over time and responds to changes in the profession. There are signs that the adoption of Agile in businesses is in decline (Lin, 2023) and market saturation of Agile and its methodologies. Such changes will impact the Agile community in different ways. It may increase competition between members and encourage more self-promotion to remain visible. Self-promotion was one of the primary factors that participants identified as lowering psychological safety.

Finally, this study used a two-phase approach to collect data on the Agile community in the first phase and then involve members of that community in disseminating that data in the second phase. This allowed us to assess the credibility of the research and develop more insights. Participants evaluated this process of shared sense-making between “Very interesting” and “Extremely interesting.” As researchers, we also found this process useful. Participants discovered some errors in the visualizations, helped generate more insights, made suggestions for improvements, and generated helpful follow-up questions for future research. This type of member checking proved a valuable way to ground research in the practical experience of professionals.

Practical recommendations

We now turn to practical recommendations based on this study. The first recommendation is to raise awareness of psychological safety and why it is essential for shared learning. A good start is the work by Edmondson (1999), although it is focused on work groups rather than communities. The current study shows that low psychological safety makes people withdraw from further engagements, may harm their mental well-being, and cause tribalism. It is also true that a portion of participants are either unaffected by low psychological safety or don’t experience it as such. However, this is a minority in this study.

A second recommendation is to invest in two factors suggested by participants of this study. First, the quality of dialogue can be improved by a commitment to keeping interactions professional and on-topic, asking questions before questioning views, using scientific evidence to support arguments, and practicing Agile values. Second, community engagement and leadership, as practiced through creating work agreements, applying moderation where possible, and standing up to behavior that lowers psychological safety together, are important.

Psychological safety is higher in face-to-face interactions. So those who want to share knowledge, may consider whether their content is more suited to face-to-face settings instead. Illustration by Thea Schukken.

Finally, psychological safety is higher in face-to-face interactions. Content creators and contributors do well in keeping this in mind. While it is easy to post a comment on LinkedIn or reply to a thread on a forum, a face-to-face interaction — in person or virtually — may make such interactions much more pleasant and fulfilling. Our study also suggests that if shared learning is the aim of a contributor, social media does not seem to be as suitable as low(er) psychological safety, which may diminish weaker voices.

Limitations

In this section, we discuss the limitations of this study. First, we do not know to what extent our sample of 143 respondents reflects the total population. For example, 30.1% of our sample consists of women. It is unclear how accurately this reflects the true percentage of women in the Agile community. The same uncertainty exists for the other properties in our sample: role and age. We addressed this limitation by using many different venues to reach potential participants. This includes several Agile conferences, posts by well-known thought leaders in the Agile community, blog posts on several well-known Agile blogs, and two industry podcasts.

Second, survey studies such as this are susceptible to self-selection bias. This causes the participants in the sample to be (more or less) different from people in the total population. This reduces the generalizability and the validity of the findings. In our study, we cannot conclusively rule out that the people who participated have experienced more examples of lowered psychological safety and were thus more likely to participate. While 4.2% of participants in our sample reported no experience with behaviors that lowered psychological safety, we do not know how accurately this reflects the number of people in the Agile community without such experiences.

Third, we defined “Agile community” as “the global group of people interested in Agile and discussing it, online or in-person.” This group with open, permeable boundaries consists of many smaller, more-or-less connected communities, but at least on a shared topic of interest. Our sample primarily consisted of people who contribute to the Agile community via LinkedIn, followed by some distance through forums and workshops, meetups, and conferences for face-to-face interactions. Because we do not know the true size of the Agile community — every place where people meet to discuss Agile — nor all the communities that make it up, we do not know if this sample is a good reflection of where most people with an active interest in the discussion of Agile can be found.

Conclusion

We defined the Agile community as a global knowledge-sharing community consisting of many smaller subcommunities. Research has shown that psychological safety is an important predictor of members' intent to continue contributing to virtual communities.

Most participants experience a moderate to high level of psychological safety for in-person interactions. For online interactions, most participants experience a moderate to low level of psychological safety. The behaviors that participants typically experience are the rejection of (new) ideas, tribalism, and not feeling heard. A small group reports no experience with behavior that lowered psychological safety for them. The most common strategy for dealing with this is for participants to stop contributing or withdraw from the community. A tenth of participants reported lowered emotional well-being. A similar-sized group reports no effect on them.

We found no significant difference in reported psychological safety by gender. However, women report substantially more dismissal of their views and tend to withdraw more from further interactions. A significant difference was found when comparing age groups, where the lowest results are reported by the age group 36–46. We found substantial differences between roles, with facilitators and trainers reporting the lowest level of psychological safety.

Participants identified three primary factors contributing to lower psychological safety: poor debating skills, dogmatism, and self-promotion. According to participants, the three primary strategies to improve psychological safety are community engagement and leadership, dialogue quality, and personal coping strategies.

Future studies can replicate this study to analyze trends and compare psychological safety between different communities. We identified several limitations to our research, which primarily impact the degree to which the findings can be generalized to the Agile community.

Whether the level of psychological safety is too low or good enough is a normative one that is up to the community. However, research shows that the intention to share knowledge is lower when psychological safety is low. If we combine this with observed differences in gender, role, and age, knowledge sharing may be harmed, lowering the diversity of those contributing.

Performing this two-phase study, analyzing the data, and writing this report took 143 hours or 17.8 full work days. While that is a lot of time, it aligns with our mission to bring more evidence-based discussions to our professional community. If you believe research like this is important, please contribute to our Patreon at https://www.patreon.com/liberators.

We are in the process of preparing this study for peer review in an academic journal. This typically takes between 1 and 2 years. The anonymized data will be published alongside the academic publication.

References

Boswell, W. R., Boudreau, J. W., & Tichy, J. (2005). The relationship between employee job change and job satisfaction: the honeymoon-hangover effect. Journal of applied psychology, 90(5), 882.

Chew, R., Bollenbacher, J., Wenger, M., Speer, J., & Kim, A. (2023). LLM-assisted content analysis: Using large language models to support deductive coding. arXiv preprint arXiv:2306.14924.

Edmondson, A. (1999). Psychological safety and learning behavior in work teams. Administrative science quarterly, 44(2), 350–383.

Edmondson, A. C., & Lei, Z. (2014). Psychological safety: The history, renaissance, and future of an interpersonal construct. Annu. Rev. Organ. Psychol. Organ. Behav., 1(1), 23–43.

Hayashida, T., Nakamura, Y., Choi, H., & Arakawa, Y. (2024, March). Privacy-aware Quantitative Measurement of Psychological State in Meetings based on Non-verbal Cues. In 2024 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events (PerCom Workshops) (pp. 433–436). IEEE.

Kostovich, C. T., O’Rourke, J., & Stephen, L. A. (2020). Establishing psychological safety in simulation: faculty perceptions. Nurse education today, 91, 104468.

Lin, B. (2023). Capital One job cuts signal trouble for IT labor market. The Wall Street Journal. https://www.wsj.com/articles/capital-one-job-cuts-signal-trouble-for-it-labor-market-11674251823

Nilsson, L., & Norström, W. (2024). Understanding Psychological Safety in Hybrid Work Teams: Experiences of Barriers and Facilitators.

Tai, R. H., Bentley, L. R., Xia, X., Sitt, J. M., Fankhauser, S. C., Chicas-Mosier, A. M., & Monteith, B. M. (2023). Use of large language models to aid analysis of textual data. bioRxiv.

Zhang, Y., Fang, Y., Wei, K. K., & Chen, H. (2010). Exploring the role of psychological safety in promoting the intention to continue sharing knowledge in virtual communities. International Journal of Information Management, 30(5), 425–436.