“We are putting all work items in Jira so we can make our teams’ work and progress transparent.”
— Manager in a company undergoing an Agile transformation
This sentiment of making all work visible to everyone is rampant these days. On the surface, it appears to be a step towards increasing transparency. But in Agile thinking, making all work and progress transparent is not the goal of transparency. Transparency in Agile and Lean reveals problems early so the team can resolve them or ask for help.
We often believe making the work visible is a good thing to do. After all, the Hawthorne Effect shows people, when being watched, choose to increase their performance. Many, in error, interpret this to mean more transparency means more productivity.
Often, we start the Agile journey from a traditional command-and-control, plan-focused, status-driven culture. Managers in this type of culture find comfort in plans and status transparency. They need this for operational control of how the teams work.
When autonomous Agile teams form, many managers feel a loss of visibility and control. Putting all work in a backlog management tool, such as Jira or Azure DevOps, is the typical response.
My gut instinct always warns me of the flaws in this pattern, and I am witness to the detrimental effect it has on teams. Putting work in a tool for manager visibility doesn’t result in the transparency desired. Instead, it leads to waste, hiding, reduced experimentation, and productivity loss.
Recently, I came across a research study that validated my instincts and observations. I’ll review the study with you below. Then, we will see how we can apply it to embrace transparency and enable empiricism in an Agile journey.
The Transparency Paradox field experiment
Little research exists on the impact of transparency on productivity. The popular belief assumes increased transparency enhances productivity and organizational performance. In 2012, Ethan S. Bernstein of Harvard conducted a field experiment.¹ His goal was to see how transparency affects productivity in the real world.
Factories are particularly ardent in embracing transparency and making work visible. So he chose PrecisionMobile, a mobile phone factory in China — the second largest in the world. To ensure authentic observed behavior, he embedded researchers as factory line workers.
Bernstein selected PrecisionMobile for its transparent organizational design. The floor had football field-length visibility to all lines of work. To make managers and leads easy for line workers to find, they wore bright, colored hats. As we will see, this created two-way transparency.
Productivity was essential for the survival of this factory. The competition for manufacturing contracts was intense with razor-sharp margins. To remain competitive, all line workers followed a fine-tuned standard operating procedure.
Let’s take a look at the enlightening results of this study.
Hiding in broad daylight
The study found behavior described as a “reverse Hawthorne effect.” When managers were absent, line workers devised secret methods, improving the flow of work. These tweaks were outside of standard operating procedure — a private deviance from standards.
When managers came around, line workers immediately reverted to standard, sanctioned operating procedures. They did not want a reprimand for their non-standard tweaks to the process. So when watched, they went back to a less effective method of working to blend in and avoid confrontation.
Line workers did not reveal their innovations to management. They did not want to “bear the cost of explaining better ways of working to others” or “get in trouble.”
So workers had a secret code of conduct to hide innovations from management. They found it easy to spot managers in their bright, colored hats across the open plant floor design. This allowed workers to devise a word-of-mouth, early warning system as managers approached.
“Everyone is happy: management sees what they want to see, and we meet our production quantity and quality targets.”
— PrecisionMobile Operator in the study
The behavior was visible from a bird’s eye view of the factory floor. As managers approached and observed line workers, performance degradation was obvious.
Privacy improves performance
But the experiment did not end here. Clever experimenters decided to install curtains around an entire line to see what would happen when workers were hidden from management line of sight. Would the same results occur? The hypothesis, of course, was to see if a productivity increase would result.
In effect, the curtains reduced the transparency of the line’s operation and created privacy for the line. Installation of curtains was a normal procedure for this plant to protect a line. So the workers did not assume anything out of the ordinary.
The results of privacy on the performance of the line were significant. Defect-free units per hour improved 10–15% within the first week. The private line maintained a performance lead over all 28 lines for the entire five-month study.
After the study concluded, Bernstein interviewed the line operators. They indicated the following contributors to increases in performance:
- Privacy to permit tweaks to address temporary issues without manager intervention.
- Privacy to permit new idea experiments before having to explain them to management.
- Privacy to avoid value-reducing hiding activities required when management was around.
The embedded researchers reported more benefits from privacy. Within the veil of the curtain, workers were free to cross-train and learn from each other. The line became a stronger team as they could now contribute across stations to help each other.
“If we didn’t need to hide things from the management levels, we could finish production so much faster.”
— PrecisionMobile Operator in the study
Transparency increased within the curtain. This allowed for increased collective knowledge, team cohesion, and team performance. The curtain provided safety for the line to innovate and have autonomy.
The line moved production reporting whiteboards outside of the curtain. They did this to keep management informed of their performance. And when problems arose they could not solve, the line pulled management into the curtain to help.
With performance improvements visible to management, deviance to standards was not a concern. Instead, managers became curious about what was making the line more effective. And workers from other lines came into the curtain to see what special sauce they had devised.
Privacy and transparency patterns to enhance Agile behaviors
I found many parallels from this study to pursuing transparency on the Agile journey.
Empiricism is a critical component of the Agile mindset. An experimental approach is an antidote to uncertainty in the product development world. Being transparent with experiment results lets the improvement process do its magic.
Transparency enables inspection. Inspection without transparency is misleading and wasteful.
Scrum has the three pillars of an empirical process — transparency, inspection, and adaptation. Without transparency, inspection and adaptation cannot occur; this would cripple the empirical process.
But as seen in the factory study, there is a nuance to how you use transparency. A critical distinction is who needs transparency and what needs to be transparent. Of equal importance is the notion of privacy. It is a key ingredient for enabling the psychological safety necessary for high-performing teams to emerge.
Let’s discuss how this applies to Agile teams, management, and community.
Transparency and Agile teams
Whoever manages the work needs visibility of the work and problems. In Agile, we have self-managing teams. And Scrum relies on a self-managing team.
They (Scrum Teams) are also self-managing, meaning they internally decide who does what, when, and how.
Given the team manages itself, visibility of the work and how it gets delivered is relevant only to the team. In Scrum, the Scrum Team owns the Sprint Backlog. Those who are outside of the Scrum Team do not need visibility into the Sprint Backlog.
The privacy afforded by the Sprint boundary is critical; this is the curtain for the team. This private space affords the team room to manage itself and deliver the Sprint Backlog as it sees fit. Burn-downs, burn-ups, Sprint boards, and velocity are delivery metrics for the team.
As in the PrecisionMobile study, privacy gives the team space to innovate and improve. A self-managing team does not need an outside body inspecting how well it is executing. The team owns its transparency, inspection, and adaptation.
Transparency and management
Because teams are self-managing, managers don’t need visibility within the Sprint boundary. Managers should instead shift focus to enabling great teams and removing obstacles. This is Agile Leadership, and we need managers to embrace this behavior to enable Agile teams to thrive.
Managers must create a safe environment for teams to become self-managing. This includes giving teams the space they need to innovate and improve on their own. It also means allowing teams to fail without fear of reprimand.
No Scrum Team needs a manager to be the Scrum police.
Agile Leaders engage by attending Sprint Reviews and going to the Gemba — the real place of work.² Understanding the product roadmap, release backlog, and release burn-up are crucial for managers. These are all provided at a Sprint Review or by asking the team.
Using a backlog management tool to inspect team performance is not useful. Getting out of the office and engaging with the team brings better transparency. And it helps managers identify how they can help the team thrive.
Transparency and community
As teams innovate and tweak the way they work, other teams will want to know. Having a mechanism to share innovations will spread ideas and allow other teams to benefit.
Holding frequent communities of practice are great mechanisms for sharing innovations. These can take many forms. They can include teams within a product, across products, or across the enterprise. Consider these ideas for sharing team innovations:
- Open-space format Town Halls
- Product-level cross-team retrospectives
- Enterprise-wide communities of practice
Taking it forward
Backlog management tools are often seen today as key to enterprise transparency. But this wide-open transparency can result in reduced productivity. The research study of the PrecisionMobile plant in China shows how this can happen.
Teams might feel the need to hide behavior if they fear they are being watched. This can stifle innovation and lead to waste.
So we should stop worrying about getting the work visible in a tool. Instead, we should give teams space to self-manage their work.
Making the Sprint a veil of privacy for the team allows them room to grow as a team, share knowledge, and improve.
While not intuitive, make privacy a priority for your teams. In the end, team privacy will enable a healthy environment of transparency to emerge.
Want to discuss this post in a live webinar?
Click here and register to join the author, Todd Lankford, to discuss transparency’s surprising effect on productivity. Todd will be presenting in a live webinar and Q&A on April 29th at 11:30 am EST, hosted by the great folks at Retrium.
You can read posts related to this one below:
The Best Metric for Your Product Team
Sometimes metrics can get in the way of a good conversation.
How fear can destroy empiricism
And the surprising effect taking risks has on psychological safety.
Surviving Agile Without a Status Report — Visualize the Work
Agile deprioritizes the status report in favor of promoting the self-organizing team. Agile asks managers to trust a…
- Bernstein, Ethan S. “The Transparency Paradox: A Role for Privacy in Organizational Learning and Operational Control.” Administrative Science Quarterly 57, no. 2 (2012): 181–216. Accessed March 13, 2021. http://www.jstor.org/stable/43548317.
- The Many Sides of a Gemba Walk, Russel Lindquist, isixsigma.com