Problems Large Teams face with Agile: Part 1: Team and Culture
This is part 1 of a 3 part series on the problems large software teams face when they adopt Agile. In this part we will focus on the team and culture aspects. In part 2 we will focus on the Product. And in part 3 we will analyze the business aspects.
If you are interested, you can download our Free eBook where we discuss these issues and some solutions that might work for you.
Agile assumes small self-organizing cross-functional teams that can make all decisions within the team. But, Agile does not tell us how such a team gets created and attains the state of self organization. Possibly, it’s left out of the agile literature because it is too complex and would vary for each team and so it is hard to be very prescriptive. However, it is important that each team that is trying to adopt Agile takes a hard look into this.
Here we will discuss some of the aspects that needs to be thought through and either the team structure or some cultural changes needs to be instituted to suit Agile or the Agile process itself needs to be adapted to suit the team structure and culture.
Enterprise Software companies have traditional project governance and decision process that are not captured or supported well in Agile
Currently large software companies are structured in a way that allows for cross functional teamwork only at higher levels in the hierarchy. The team members executing the projects are often at lower levels of hierarchy in the organization chart where they work isolated from other functions. Any decision that requires cross functional team work then needs to bubble up the org structure; the decision needs to be made; and then the decision needs to trickle down.
The current process deployed in these large companies assumes this organization structure and works with it. Sometimes, when trying to adopt Agile, teams ignore the cross functional aspect of the scrum team and faces some expectation mismatch downstream.
Some companies create the cross functional project teams when they start adopting Scrum without undertaking any other organizational or people management process changes. This is done partly because of the local problem solving mindset with which Agile is adopted.
Part of the reason for this is also that the idea of cross functional teams are not totally foreign to these companies. There are cases where cross functional teams are created to solve some burning issues. In some companies these teams are called tiger teams. This somewhat familiarity causes more problems. If a company was fully unaware of cross functional teams, they would sit up and take notice and maybe think of how to make it work. But due to this somewhat familiarity, often teams embrace it without fully assessing the implications. Tiger teams are typically temporary and works on high profile, high visibility, high impact tasks. Working in these tiger teams is different from perpetually working in cross functional teams in long running projects.
The call here is: do not sidestep this issue. It is important to take up this issue head on, analyze the current organization and culture, set attainable goals, and make a deliberate decision.
Career Growth Path
Today most large companies are organized by function. These functional teams eventually report into some general manager. In these organizations the titles and career growth plans are deep seated. The career growth plan that most people are aspiring for are typically: X, Senior X, Manager of X, Senior Manager of X, Director of X, Senior Director of X, Vice President of X, and so on. The job function, reporting chain, the title, and career growth plans are all aligned nicely. If project teams are supposed to be “self-organizing cross-functional teams” then one needs to come up with an organization structure and career growth plan that are aligned with the idea of self-organizing cross-functional project teams.
Career growth is also related to the ability to change jobs across companies in the same industry and also move to different industries. Personal development of reusable skills and title and recognitions that identify these skills are necessary to secure the ability to switch jobs. The existing culture of narrow functional expertise development supports this job change aspiration very well.
The Role of the functional manager
The role of the manager is not well defined in Scrum. Large software companies using traditional project management had big roles for the functional managers in projects (in addition to the people management role). Introduction of Scrum in these teams creates a confusion. Does the functional manager take up a new role or does the functional manager recede to a people-manager-only role?
It is important to have an open discussion at the onset to reduce uncertainties during the transition.
In large software companies, teams are created. Teams and organizational structures are almost synonymous in these companies. Even when occasionally, cross functional tiger teams are formed, they get created by a manager. So this concept of self organize is outlandish to these teams. Consider your options carefully and make a deliberate choice. Make sure this key concept is well understood and accepted by all in the beginning.
Ask yourself the question: how do teams work? Look around yourself in the teams you work on. Study successful sports teams. You will see leadership everywhere. Large companies are successful companies. They achieved this success through leadership of their team members. In these companies leaders are identified and established with titles conferred on them and leadership is expected from them with the roles and responsibilities given to them. In Scrum they are all equalized as “Team Members”. Scrum still needs leadership for it to function but it is not as explicit in evoking it. This creates a state of confusion and needs to be dealt with early for Scrum to succeed. Self organize cannot be taken to mean no leadership needed.
In Agile and Scrum there is a lot of emphasis on the “Team”. The Team organizes itself and takes decisions. In traditional organizations this is done by a designated person on behalf of the Team, and these individuals are accountable.
In Agile each team member is accountable to each other for different things. It is important to make sure this is understood by everybody. More importantly, for teams starting up with Agile it might look like some accountability lies with the “Team” (because of the emphasis on the “Team” on various things). It is important to make sure that in practice a “Team Member” is accountable for each major work or decision.
Power and Responsibility
In any team power comes from two sources: “organizational power” and “knowledge power”. In any large company these sources of power are clearly identified by the organization structure and title. Currently, in large companies power and responsibility are aligned. When companies adopt Agile they either dismantle the earlier organization and create the new structure based on Scrum teams, or they keep partly or fully the earlier organization and titles and overlay the Scrum based teams. In either case it is important to make sure the power and responsibility equation is understood by all and everybody is onboard with it.
Skills level, experience, maturity
Agile does not talk about different skills level, experience, and maturity of “team members”. Traditionally, there have been roles for architects or team leads. It is important to appreciate that these roles are not present just to pump somebody’s ego. These are important functions in the software development process. One needs to think about how these roles function in Scrum teams. Similarly, not all software developers come with the same skill level, experience and maturity. This needs to be kept in mind when creating teams (even when they are self organized), and when doing sprint planning.
The first among equals
When trying to create (or self organize) many small scrum teams in a large project, there might be some people whose skills and ability are needed by multiple Scrum teams. Scrum does not provide a very good solution for this. Scrum assumes independent Scrum teams. It is important to think of a framework for this and still keep Scrum teams as independent as possible.
Product Owner vs. Product Manager
Large software companies typically have a Product Management team. Scrum teams should have a Product Owner. The roles are somewhat overlapping but different. So when adopting Scrum, a critical dilemma for teams is how to rationalize these roles. The options are:
- Convert all Product Managers to Product Owners
- Keep both Product Managers and Product Owners
Before we can analyze the above choices, let’s try to understand the difference between these roles. Both Product Owners and Product Managers are expected to straddle and bridge the external business side and the internal technical side of the organization. However, in large companies using traditional Project Management, Product Managers are typically more business focussed with basic understanding of the internal technical details. The gap is typically bridged by Project Managers, Architects, Team Leads. On the other hand Product Owners are expected to have deeper technical and product knowhow and are expected to be available to answer detailed questions on a daily basis.
So when thinking of the first option of converting Product Managers to Product Owners, keep in mind the re-skilling needed and the career aspirations and goals in mind. Also, be aware that the number of product owners needed in a large team would typically exceed the number of Product Managers present in the team. While there are some cases where this strategy can work, for large complex products with a complex business model this will likely fail. Fulfilling both roles might be too much load on one person, even if you can find someone with the skills and willingness to fill both roles.
When thinking of the second option of keeping both Product Managers and Product Owners, there are two issues that needs to be solved:
- how would you create the team of Product Owners, and
- how would the Product Owners work and collaborate with the Product Managers.
Setting up the right collaboration and communication framework between the Product Managers and Product Owners is critical for success in this case.
Scrum Master vs. Project Manager
The issue here and the ensuing confusion is similar to the one we discussed above about Product Owners and Product Managers. Large software companies have Project Managers and Program Managers. When adopting Agile, again the choices in front of companies are:
- Convert all Project Managers to Scrum Masters
- Keep both Project Managers and Scrum Masters
Project Manager’s role is a leadership role to manage all aspects of the project. It’s a soup to nuts kind of responsibility starting with ideation to making sure the business outcome of the project is met.
Scrum Masters are more concerned with the scrum team only. The role is to coach and make sure the process is working and to remove impediments. While they are similar, the roles and responsibilities and scope is quite different.
There will be cases where one person can perform both roles and there can be situations where keeping these roles separate is for the best. This is a critical judgement teams must make when embarking on Scrum adoption.
Transparency and Regular Updates
Scrum attempts to add a higher degree of transparency to projects. This definitely should be seen as a good approach. However, in some organizations this level of transparency might look like intrusion. It would be useful to first think about the culture of the team, and understand how much of a culture shock this will bring about. Open and explicit discussion on this topic where everyone can contribute and appreciate the positive impact of the increased transparency should help smoothen the transition path.
In the Scrum meetings one is expected to talk about the previous day’s accomplishments, the next day’s plan, and any item that is blocking progress. In some cases where the work is highly complex or creative it might be difficult to do this very well. It is important to make sure that the culture in scrum meetings does not force people to “invent updates” rather than state the reality and focus on the work.
Parts of software development work, especially when broken down into small enough chunks, are reasonably straightforward and can be pre-planned with certain degree of confidence. However, often there are pieces of work that are either highly creative or highly complex where it can get difficult to describe what one will do next. Often the last day’s achievements can also be difficult to state in short. Take the example of working on a difficult to reproduce, mysterious defect in a large software product. The update for many days could be “I was debugging yesterday, and I will be debugging today”, until the day a breakthrough is achieved.
Sometimes, one can state what was done yesterday. This can encourage others to chime in with suggestions. However, often it is much harder to state in simple form the next day’s plan. Good debugging is often a large decision tree that is not fully mapped out in the beginning. The team and Scrum Master must use their judgement on the level and depth of updates that makes sense for each work item.
Generalists vs. Specialists
Agile or Scrum as such does not have anything specific on this topic. However, some of the processes does encourage creation of a team of Generalists. One needs to assess where their team currently falls in this dimension, what the product demands, and the future goal of the team.
Geographical distribution of teams
Large Software Companies have large globally distributed teams. Often the way the teams got created resulted in different functions being present in different location. As an example, multinational companies grew their presence in countries like India with only engineering teams. Even within engineering, often, different module teams gets created and grows in different locations. So the current situation is such that it is not possible to create a collocated cross functional team in India or many such other locations.
In the next blog in this series we will discuss the product and how that should influence the choice of process.
This article was first published in Wizergos Blog.