Leading Agile Development

What It Takes to Make Agile Work for You

Agility is the speed with which an organization can exploit opportunities and respond to challenges. It is a priceless competitive advantage for businesses that achieve it. In today’s ultra-competitive world, business agility requires a mastery of computing and software technology. To achieve that, we must also master Agile software development.

What Is Agile Development?

In 2001, seventeen very experienced software developers met at a resort in Snowbird, Utah, to discuss lightweight development methods. The Manifesto for Agile Software Development was the result. Both the meeting and the manifesto were responses to professional developers’ deepening frustration with the heavyweight waterfall development model then in vogue. To be considered Agile, development models should adhere to the principles behind the Agile Manifesto.

The Manifesto promotes these values:

1. Tools and processes are important, but it is more important to have competent people working together effectively.
2. Good documentation is useful in helping people to understand how the software is built and how to use it, but the main point of development is to create good software, not documentation.
3. A contract is important but it is no substitute for working closely with customers to discover what they need.
4. A project plan is important, but it must not be too rigid to accommodate change as the understanding of the problems and the solutions evolve.

Waterfall versus Agile

Way back in 2000, the consulting firm, for whom I then worked, was engaged by a rapidly growing Florida telecom to help them understand why they were having difficulty building and deploying the custom software systems that were so critical to running their business.

As I started my evaluation, I noticed that they were rigorously following the waterfall model as specified in a methodology manual contained in twelve 2-inch binders. They had purchased it, along with the training, from what was then one of the prestigious “Big 8” professional services firms.

They were not particularly thrilled when, at the end of my 2-week evaluation, I reported in depth on why and how that methodology was a major contributor to their software development problems.

The waterfall model originated in the manufacturing and construction industries, where highly structured physical environments meant that design changes became prohibitively expensive much earlier in the development process than one might wish.

Because of that economic reality, the waterfall model expends a large proportion of its time and resources to defend against change. In the 1980s, the U. S. Department of Defense virtually mandated, with Department of Defense Standard 2167A, the use of the waterfall model by all its software contractors.

When we developers adopted the waterfall model back then, it seemed to make sense, and there weren’t many other well-understood alternatives suited to knowledge-based creative work like software development. We just hadn’t yet learned enough.

The waterfall model describes a production line for software development, as if software were a solid physical thing. It is a very inefficient model for the discovery, learning, and experimentation that are the foundations of building software. Waterfall defends against change, while Agile is optimized to respond effectively to the inevitability of change.

Agile Development Frameworks

The two most popular Agile frameworks, Scrum and Kanban, originated from the Japanese Lean Manufacturing movement, and derive from W. Edwards Deming’s ideas about quality and efficiency. Those ideas helped propel Japanese auto manufacturers, like Toyota, past their more traditionalist American rivals such as Ford and GM, and so confirmed the disruptive potential of Agile in business.

An objective, of both Lean and Agile, is to put the most effort into the things that deliver the most value for the customer and thereby for the company. Both Agile frameworks emphasize that objective.

Though these two frameworks predate the Agile Manifesto, they were already tuned to the lean and agile philosophy advanced by it. It was a relatively straightforward process to create versions of Scrum and Kanban that were suited to software development. Ultimately, their rapid adoption by proponents of Agile confirms the fit.

Though I personally prefer Kanban, and I believe that Scrum has issues, either framework can help you create an Agile environment. I’m an old developer and moved to Agile because I believed that I had learned why the waterfall approach was not working. Kanban better matches my mental model of the shape and properties of development projects.

Kanban, with tools like Jira, was an easier mental leap for me, and I found that its artifacts made it easier to report status up the traditional chain of command. It is purely a matter of personal taste; six of one, a half a dozen of the other.

If you would like another perspective on how the two frameworks compare, you might enjoy reading Chunting Wu’s post, How Kanban Works and Why I Prefer It Over Scrum.

In the final analysis, what really matters is how well you use the framework you do choose.

So What Does It Take to Be Agile?

Implementing Scrum or Kanban won’t make your organization Agile. They are just tools. Agile is a state-of-mind, and a way to think about solving problems. It is an organizational mindset, and a way for teams of human beings to work more effectively together to achieve common goals. It is not magic, and it requires effective leadership and communication to succeed.

I am using the term leadership, and not the term management, purposefully. Not all managers are leaders, and not all leaders are managers. Implementing cultural change within an organization requires leadership. A manager without leadership skills is a bureaucrat with authority. That is the kiss of death when trying to become Agile.

There are videos, books, and training on Scrum, Kanban, and becoming Agile. This post is not intended replace them. If you sincerely wish to make your organization Agile, I would recommend you start with the videos and books, and then, after you have learned enough, select an Agile framework.

That’s the point at which you could engage a firm, that specializes in Agile training, to help you plan and execute your transition. I do not recommend converting your whole development organization to Agile practices simultaneously. Start small, get something working well, and then expand its scope.

Most importantly, start with a manager who is a capable leader. Without effective leadership, Scrum and Kanban can quickly devolve into form over purpose and fail to actually become Agile.

An Agile leader’s role is to communicate objectives and direction, identify and remove obstacles, provide guidance when needed, mediate the inevitable disagreements, and enable the individual players to succeed. Above all it is to communicate effectively both horizontally and vertically. It is not to micromanage projects or people.

What Are the Most Common Pitfalls When Implementing Agile?

Over the last 15 years, as Agile development has exploded in popularity, I’ve observed some common pitfalls that can make transitioning to Agile more difficult. Avoiding them will make your Agile journey go a lot more smoothly.

1. When you bite off more than you can chew. I’ve witnessed some very dramatic failures when organizations stopped everything, held immersive framework training for everyone, gave the requisite pep talks, and then on the next Monday, all of development, and development’s customers, were expected to use the new framework and miraculously be Agile. That never works. Becoming Agile is a journey, and a framework requires practice and experience to understand and use properly. It is better to start small with one team so you can learn how to make the framework function within your organization and culture. Then that team can mentor the next team as it transitions to the framework, and so on and so on. Productivity initially declines while a team is learning how to use an Agile framework. If you try to convert all of development at once, gridlock can effectively shut down development productivity for months. That’s not good for the company, and it’s definitely not good for careers.
2. When framework events sap productivity and waste time better spent on actual productive tasks. That is often due to misunderstanding the purpose of an event, like a daily stand-up, and can cause events to devolve into overlong meetings rather than brief time-boxed checkpoints. Events should be used only to communicate status and identify issues. Issue resolution should result from a separate activity to be scheduled among the involved team members.
3. When framework practices are not implemented in the spirit of the Agile Manifesto, they can become a form of micromanagement and create the same dysfunctions that the practices were intended to cure. The framework is not the objective, development Agility is.
4. When a framework implementation assumes that the effort required for completing a task can always be accurately estimated, even though that is rarely the case. I have found that Kanban deals with this more naturally than do Scrum sprints.
5. When a framework is used as a justification for ignoring sound software engineering principles such as software architecture, requirements, and design. A foundational Agile principle is that the best architectures, requirements, and designs emerge from self-organizing teams, not that they aren’t necessary. Continuous attention to technical excellence and good design enhances agility. Marginalizing software architecture and engineering decreases software quality. Decreasing quality can be one of the most expensive things that we can do wrong.
6. When a framework is turned into a hamster wheel for developers. Agile’s highest priority is to satisfy the customer through early and continuous delivery of valuable software, and its frameworks are intended to promote sustainable development. Its sponsors, developers, and users should be able to maintain a constant pace indefinitely. But sometimes the metrics, used to measure iterative process improvement, become an end unto themselves, and sustainable development gets replaced with an unsustainable hamster wheel, racing to go faster and faster, and burning out developers in the process.
7. When a framework is perceived to be a form of, rather than an alternative approach to, project management. Agile frameworks are a state-of-mind and a way to think about solving problems. They are an organizational mindset, and a way for teams of human beings to more effectively work together in order to achieve common goals.
8. When an organization assumes that a framework replaces the need for leadership. Agile promotes the idea of self-organizing teams. That does not eliminate the need for leadership at a business and a team level. Leadership is even more critical as the development culture transitions into an Agile culture. The frameworks replace project management, not project leadership.

Wrapping up

Business agility is very often critical to business success. The ability to quickly exploit opportunities and respond to challenges usually requires agile and effective use of digital computing technology.

Agile software development values, as articulated in the Agile Manifesto, are intended to help iterate more quickly by focusing resources on the things that create the most value for the business, while minimizing the resources expended on things of lesser value.

Creating value takes more than using an Agile framework, it also takes business acumen and leadership to truly enable the people who do the real work.

“Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.” is an important principle of Agile development, as well as a good definition of the role of effective Agile leadership.

Thank you.

Well worth reading: 4 Common Misconceptions About Agile Transformation

In memory of William Edwards Deming (October 14, 1900 — December 20, 1993) who was an American engineer, statistician, professor, author, lecturer, and management consultant. Educated initially as an electrical engineer and later specializing in mathematical physics, he helped develop the sampling techniques still used by the US Department of the Census and the Bureau of Labor Statistics.

Deming made a significant contribution to Japan’s reputation for innovative, high-quality products, and for its economic power. He is regarded as having had more impact on Japanese manufacturing and business than any other individual not of Japanese heritage. Despite being honored in Japan in 1951 with the establishment of the Deming Prize, he was only just beginning to win widespread recognition in the United States at the time of his death in 1993.

The two most popular Agile frameworks, Scrum and Kanban, originated in the Japanese Lean Manufacturing movement and derive from Deming’s ideas on quality and efficiency. In many ways, Deming was the father of “Agile”. I know that his ideas had an enormous impact upon how I learned to structure and implement software projects when I was starting out.