Contrasting the Waterfall Model, Agile, Lean and DevOps

Freddy Yumba
May 5, 2016 · 17 min read

Continuous Delivery, Continuous Integration and Culture

Josef Langerman

Introduction

However, in order to improve IT service performance and to give organizations competitive advantages, we need more than just simple agile methodologies. Gartner declared that “DevOps movement was born of the need to improve the agility of IT service delivery and emphasizes people and culture and seeks to improve collaboration between development and operations teams while seeking to remove the unnecessary impediments to service and application delivery by making use of agile and lean concepts” (Wurster, et al., 2013). For this reason, DevOps can be considered as the integration and application of different software development methodologies, operational processes and social psychological beliefs for transforming IT service delivery. In this perspective, DevOps is a new way of thinking, a spirit and a philosophy for transforming organizations.

We compare agile methodologies to lean concepts and practices so as to understand the relationships between these two ways of working and see how they encourage continuous integration and continuous delivery of IT services and how the combination of these two methodologies create a new culture, philosophy and spirit for development and operations teams. When it comes to telling which software development methodology is good, better or the best, it all depends on the view of the proponents of the methodology, the type of system to be developed and the culture of the organization.

The waterfall model vs the agile way of thinking

Figure 1. The waterfall model (Sommerville, 2011).

The waterfall model, also referred to as the software life cycle is an example of a plan-driven approach to software development. Ian Sommerville in his book entitled “Software Engineering” supports that the waterfall model is a linear approach to software development where the results of each phase is one or more documents that are signed-off and that the following phase should not start until the previous phase has finished. He also supports that in the waterfall process model, you must plan and schedule all the process activities before starting work on them (Sommerville, 2011).

While the waterfall process model focuses on following well-defined plans and detailed documentations, agile is considered by many IT professionals as a way of thinking and not a methodology. Is this true? Well, a group of software practitioners advocated a set of principles based on best practices and their previous success and failure experiences with many software development projects regarding what works and does not work. Each one of these highly-experienced professionals had their own different philosophies or way of thinking about how they approached software development. They developed the manifesto for agile software development which states the values and principles of agile software development philosophy (Misra, et al., 2012).

As we read from the above quote, the individuals who created agile practices and principles termed the output of their thinking as a philosophy (a way of thinking) and not a methodology. Philosophy informs everything we do in a method (Roberts, 2012).

So why is agile a philosophy and not a methodology?

image by David Bain

In my humble opinion, it is because agile principles denote the methods used in agile practices and these principles or methods result from critical empirical thinking which is a philosophy subjective to individuals. Another reason why agile is not a methodology but a philosophy is that agile does not refer to a single method but encompasses different software development methodologies and IT project management practices such as scrum, extreme programming practices, pair programming, etc.

Agile manifesto and principles vs the waterfall process model

Image by brane

You cannot compare agile software development to the waterfall process model without mentioning the agile manifesto and principles. The agile manifesto states the values and principles of agile software development while broadly comparing them to what is believed to be the primary focus of a plan driven approach to software development. The following is the agile manifesto as literally stated by its 17 authors (Beck, et al., 2001):

“We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:

Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan

That is, while there is value in the items on the right, we value the items on the left more.”

Thus, the waterfall model is based on the spirit of strict conformance to executing software development stages in a linear fashion from A to Z due to the assumption made that requirements can be clearly specified prior to developing the system. This had worked with the development of operating systems, embedded control systems and large and complex systems where a detailed plan is required. In contrast, the spirit of agile philosophy relies on an incremental approach to software specification, development and delivery (Sommerville, 2011). Agile is the driver for success and innovation and emphasizes individuals, working software, customer collaboration and rapid responses to customers’ needs by accommodating their later changes to requirements throughout the development processes.

When can I choose the waterfall process model over agile methods?

While agile methods are being widely used by individuals, not all projects are suitable for agile methodologies. The reason why proponents of the waterfall model argue that agile methodologies degrade the structure of many systems is because some agile practitioners do not want to admit that they can also make use of the waterfall model in software development. Safety critical systems and embedded control systems require prior adequate and consistent planning of what have to be included in the release of the system and how the system will operate. For instance, making changes to the initial design of an aircraft control system in the later phases of the development process may result in the introduction of bugs that can comprise the reliability and safety of the system under development.

Despite the fact that agile methods such as scrum which leaves the software project scope flexible to accommodating later changes to software requirements, there is still a little bit of planning. The waterfall model depicts the traditional SDLC. Every software development methodology makes use of the SDLC but in a different way. While the waterfall model follows a linear approach to executing the SDLC, agile methods rely on an incremental approach to software specification, development and delivery.

Why choose agile way of working over the waterfall model?

Courtesy of MedDigital

The twelve principles of agile software development stated in the agile manifesto describe in broader details the reasons why one may choose to follow an agile approach rather than the waterfall plan-driven approach to software development. Agile practices create an organizational culture that gives importance to customers. In order to explain the main reason why one may choose agile over the waterfall model, we compare agile principles to the waterfall model as follow:

  1. While software delivery in the waterfall process model is toward the end of the contract, agile methods focus on satisfying the customer through early and frequent releases of valuable software integrating the customer’s prioritized functionalities.
  2. While later changes to the system specifications in the waterfall model is difficult to integrate as this may involve redesigning the all system, agile methods welcome changing requirements even late in the development of the product.
  3. While the waterfall model places emphasis on documentation, agile methods focus on frequently delivering working software to customers.
  4. While development teams work in isolation away from business people, agile methods encourage collaboration between development teams, business people and customers.
  5. While the waterfall model places importance on processes and tools, agile methods focus on motivating individuals by building a culture of trust to get the work done and providing all the supports needed by the team.
  6. While the waterfall model uses documentation to communicate the plan of what has to be done in a software project, the advocates of agile methods believe that face-to-face conversation is the most efficient way of conveying information.
  7. While documentation is used as a metric to assess the progress of the project, agile advocates believe that working software is the primary measure of progress.
  8. While processes of the waterfall model lead to ineffective communication between development teams and customers, which is considered as one of the main problems that lead customers to being dissatisfied, agile processes create a culture that promote sustainable development by encouraging collaboration between sponsors, developers and users throughout the development life cycle.
  9. While the waterfall model emphasizes on following a well-established plan which does not keep in mind potential later changes to the system requirements, agile methods advocates believe that best designs emerge from self-organizing teams.
  10. Finally, while the waterfall model put emphasize on improving development processes and tools, agile advocates believe that success is achieved when individuals assess their performance regularly and adjust their behavior accordingly.

Agile or lean for improvement? Maybe both for success and innovation?

  • Poor quality of product
  • Low morale and employee engagement
  • Unsuccessful projects
  • and high cost of operations

All these result in unhappy customers and low profitability. These are the problems that lean and agile solve. These ways of working solve these workplace problems by developing a culture of collaboration and continuous improvement with the help of simple practices that change behaviors. They improve the culture and the fabric of the way for ensuring fast delivery of tangible results from day one. Agile started off as a software development way of working and lean started as a production and operation system that helped Toyota overtake GM and become the number auto company in the world (Abernathy, 2013).

Moreover, Phil Abernathy supports that agile and lean are both based on the same organizational values of trust, respect, accountability and honesty. Above this, agile and lean are based on a common set of principles of collaboration, iterative delivery, continuous improvement, empowerment and transparency. Agile and lean have a set of practices that are focused at all times on delivering customer values. This creates the fabric leadership behavior and determines the culture and success. Agile and lean are ways of working as opposed to methods. Together they give the necessary understanding to continuously improve. When we combine agile and lean we get operational efficiency, project excellence, product and service quality and a great working culture that attracts the best people and gives a competitive advantage that is tough to imitate or beat (Abernathy, 2013).

Application of lean and agile

After reading about the origin of the application of lean and agile, it is time to know why individuals are no longer just making use of agile methods but combining them with lean practices. In the following section, we analyze the transition from agile to lean and how lean practices are used to improve agile methods.

From agile methods to lean principles and practices

What is lean thinking? Womack and Jones define lean as “a way of thinking that allows companies to specify value, line up value creating actions in the best sequence, conduct these activities without interruption whenever someone requests them, and perform them more and more effectively” (Womack and Jones, 1996). As with agile methods, lean thinking comes into practice through the integration of a set of principles and practices. Mary and Tom Poppendieck realized that the lean principles that was used to improve manufacturing and production could be adapted to fit software development. The lean principles relevant to software development are (Poppendieck and Poppendieck, 2003):

  1. Eliminate Waste
  2. Build Quality In
  3. Create Knowledge
  4. Defer Commitment
  5. Deliver Fast
  6. Respect People
  7. Optimize the Whole

Identification and elimination of waste from the process with respect to customer value is the primary and guiding principle of lean way of working (Wang, et al., 2012). For this reason, while agile methods have improved software development with scrum and extreme programming practices, lean principles improve agile practices. As the primary principle of the lean way of working is to eliminate waste, it is supported that “Lean thinking forces teams to eliminate anything that is not adding value to the product or service development process. Lean organizations try to avoid unnecessary meetings, tasks and documentation. They also try to eliminate building things that are not needed, which can be challenging because nowadays, market needs change more rapidly than ever” (Chan, 2013).

Thus, when we say from agile methods to lean principles and practices, what we actually mean is not that agile methods are no longer being used by individuals but what we mean is that agile methods are integrating lean practices and principles so that these methods can scale to organizational level by improving all the development and production system operations. In the following section, we explain how lean principles are used in agile.

Lean within agile

  • People centric approach
  • Empowered teams
  • Adaptive planning
  • Continuous improvement

On top of the commonalities between agile and lean ways of working as identified by Chris Sims, we can also add collaboration, continuous integration, simplicity and fast delivery of product and services. However, if we read from the evolution of agile practices, we will see that agile methods were influenced right at their beginnings by lean principles.

Thus, we use Kim Chan’s statement to conclude this section by saying that “lean and agile share many of the same principles and many agile principles are borrowed from lean thinking” (Sims, 2008).

Difference between lean and agile

Transforming IT service delivery with DevOps by using agile and lean practices

For this reason, a new culture, corporate philosophy and way of working is emerging. This way of working integrates agile methods, lean principles and practices, social psychological beliefs for motivating workers, systems thinking for building complex systems, continuous integration and continuous improvement of IT products and services for satisfying both customers and production and development teams. This new way of working is DevOps.

What is DevOps?

  1. Adam Jacobs in a presentation defined DevOps as “a cultural and professional movement, focused on how we build and operate high velocity organizations, born from the experiences of its practitioners”. This guru of DevOps also states that DevOps is reinventing the way we run our businesses. Moreover, he argue that DevOps is not the same but unique to the people who have practiced it (Jacobs, 2015).
  2. Gartner analysts declare that DevOps “… is a culture shift designed to improve quality of solutions that are business-oriented and rapidly evolving and can be easily molded to today’s needs” (Wurster, et al., 2013).

Thus, DevOps is a movement that integrates different ways of thinking and different ways of working for transforming organizations by improving IT services and products delivery.

How to successfully integrate DevOps culture in an organization?

  1. Cross-functional teams and skills
  2. Continuous delivery
  3. Continuous assessment
  4. Optimum utilization of toolsets
  5. Automated deployment pipeline

We are DevOps when we are:

  • Lean (waste elimination and value stream mapping)
  • Making use of the best social psychological believes to motivate people and manage our organizations.
  • Making continuous improvement of products, services and processes
  • Creating a culture that invites and encourages development and operations teams to work together.
  • You can add on this list based on your own professional experience.

Continuous integration and continuous improvement are one of the drivers of the DevOps culture. In order to understand how DevOps as an organizational culture is practiced through agile and lean principles, we discuss in the following sections, continuous integration and continuous delivery.

Continuous integration

The scrum approach to software project management encourages continuous integration throughout a sprint. As developers are working on a product backlog, they continuously integrate their codes in a shared repository every day. This integration is facilitated by the use of source and version control software which allows developers to work in different machines even locations while still integrating their codes into the same repository that can be used later to deploy the entire system. Continuous integration is a must for fast and frequent delivery of valuable software.

DevOps and continuous delivery

However, continuous delivery demands continuous integration for valuable software to be delivered to customers anytime as it may be requested. In order to make this happen, DevOps put emphasis on automation. Automating the deployment pipeline is a must in DevOps as continuous delivery depends on it. Thus, DevOps is all about being agile and eliminating waste by lining up value adding activities and executing them in a sequence. As DevOps aims to transform the way organizations develop and deliver IT products and services, it needs agile principles to do so. Continuous integration and continuous delivery of valuable software are the keys to delivering products and services to customers more frequently and satisfying the customer with valuable software at a click of a button.

Difference between DevOps and Agile

Relationship between DevOps, agile and lean

Conclusion

Waterfall or agile? this depends on the type of system to be developed and the culture of the organization. Agile and lean are based on some commonly shared principles and values that emphasize people by encouraging customer collaboration and fast delivery of valuable software through continuous integration and continuous delivery. However, while agile methods are widely used as an approach to managing software development (scrum) and as a software engineering practice (extreme programming), lean principles were originally designed for improving production systems operations. Despite the difference in terms of application, agile methods use lean principles.

If you enjoyed reading this article, please click the ‘like’ button, leave some comment, share and recommend it to others.

Freddy Yumba

Written by

A type of software developer that connect business ideas and technology to bring innovation.