Keywords: Waterfall, Agile, DevOps, Lean, Software Development

Introduction

Software development processes are gaining attentions as a results of software itself being highly adopted and integrated in our lives. The demand for software require us to review time and again how we respond to this demand, how we deliver products. The discussion in this paper is grounded around the traditional waterfall, lean, agile and DevOps processes and methodologies. Each of these methodologies is discussed with reflections mainly on the how these methodologies impact organisations or team culture, and how we may continuously deliver and integrate our products. The nature of the discussion follows the evolution flow, as we first look into the traditional approach, and continue through the “modern” approaches. This discussion highlights pros and cons of each of the methodologies, while giving a picture of when a particular methodology may be suitable. Factors such as but not limited to the nature of the project, organisational structure, values, culture, goals and objectives, size of the project and the organisation, resources, time and most importantly the market narrows down the suitable approach. Now let’s evaluate the waterfall approach. The subsequent discussion introduces the traditional waterfall approach.

Waterfall

This is the old fashioned way of doing things, almost as old mathematics, since it is a sequential process. In its nature, which is why I find it similar to mathematics, when solving an equation, step n depends on what was done on step n — 1. Moreover, you may attempt to take on step n + 5 right after completing step n skipping two to four, at times you may get away with it. However, you’re most likely to fail, or your n + 5 will be bigger in size an effort making up for two to four, of which is the same as doing them either way, order matters.

The water fall model was documented for the first time in 1956 by Benington, H.D. (1956). Its focus on requirements and enforcing analysis prior to design and implementation put it above other methodologies at the time and in some cases in this day and age. Figure 2 shows the very first recommended model which has evolved to what is illustrated in figure 2. The waterfall model was updated in 1970 by Royce, W.W. (1970).

Figure 2 illustrates the waterfall approach model (black lines). This model is fairly easy to understand, and quite simple to implement. A phase, e.g. Analysis has to be completed before we may move on to design phase and so on. Moreover, once a phase has been completed it is never revised (Arora & Arora, 2016). However, it is possible to employ feedback loops in the approach. Problem is, it gets complicated (Arora & Arora, 2016).

Going back to how this approach is similar to solving mathematical problems, the very most initial step is important and from there onwards the most important step will be the one currently on. Now, what is the first phase, Analysis? Well yes, which is just the understanding of the problem. Keeping in mind that there are cruel or maybe just terrible lectures out there, who might just structure the problem in to bring disorder in your life for that moment. The model relies on clients presenting the problem in a manner that will be understood meaning that the client knows very well what they want. Talk about a blue moon.

In industry IT projects are legendary for being delivered late, or not delivered at all, and when delivered they are likely over budget (Bassil, 2012). Surely you can easily recognize some of the shortfalls of the traditional approach from the previous discussions. The major shortfall or rather the easily recognisable one is the lack of flexibility when responding to change (Mahalakshmi & Sundararajan, 2013). Inflexibility will have a negative impact on customer consummation of which affects profit. Not to say changes are never implemented whenever a client presents change in requirements. Contrariwise, client will have to wait for the initial agreed upon requirements to be implemented, otherwise starting over, and through away the stone and cast requirements on a new stone again. Clients are involved in the begging when requirements are gathered and analysed, and at the end when the “solution” is delivered which is when they can now evaluate and provide feedback, and not so during the project development (Mahalakshmi & Sundararajan, 2013). Furthermore, there is a greater likelihood of not getting everything right during a particular phase, even if the job was done, reviewed by at least three people (Kumar, Zadgaonkar, & Shukla 2013). This implies that phases inherit problems from the previous phase. In other words, we would be digging a grave, for ourselves of course. These shortfalls of presents a higher risk and there is a lot of uncertainty in the phases of the approach (Kumar, et al, 2013).

Inherited phase issues, delays and over spending accumulates from one phase to the next. This easily leads to project failure when not addressed and resolved quickly. Furthermore, idling of phases waiting for the completion of the one before ensures delays in project delivery (Bassil, 2012). Well that’s just the one of the cheeks of the waterfall approach, let evaluate the other on since this approach is highly adopted particularly by big organisations.

This approach may not be perfect but it works for most organisations, obviously with delays and the like there and there. Its characteristics of being linear and sequential, makes it easy to follow, and simple to implement and manage (Arora & Arora, 2012). Moreover, a particular phase does not require all roles to be active, for instance, developers don’t necessarily have to be part of the requirements phase which mean minimal resources are used at a time, efficient and potentially effective. Additionally, a phase embodies and drives focus, there is no design taking place during development phase and vice versa (Mahalakshmi & Sundararajan, 2013). These phases do not overlap. Furthermore, each phase produces a rich documentation of the work carried out and end results thereof. Properly documented requirements, leads to accurate analysis and design, and an easy going implementation which will be testing and maintenance friendly. High and rare quality. It is a common believe that this approach is suitable for large and critical projects.

These large organisations have common qualities. The biggest common quality is management structure. Some might follow the matrix stricture, however, it is never pure matrix, the root is a hierarchy. The nature of the hierarchy is embedded the need for control, bit of bureaucracy. Moreover, these organisations have significant amount of resources, of which can be easily mismanaged.

Organisational structure and the approach adopted have a greater impact on the organisational atmosphere. These are variables to the overall organisational culture. Considering the bureaucratic nature in the hierarchy structure and the nature of the traditional approach, can you imagine the culture of that organisation? Now, do you think generation Y, the millennials would fit, grown and enjoy such environment? In case you don’t know who millennials are, we are that group you find in graduate programmes, we don’t really remember era before internet or any cold war memories and closest event to that is the 9/11 of which we weren’t adults at the time, we are diverse, and with regards to our careers, money comes after freedom, growth, learning, recognition and social (Goldman, 2014). Moreover, we are tech-savvy. We are likely to fall for transparent, flexible and collaborative environment than for a big brand (Abbot, 2013). We are connected, and our friends’ opinions matters making big brands no so big and small brands potentially big. We are current and the way we go on about doing the thing isn’t suitable for the traditional approach as we are multitaskers and expect variety in our job.

As millennials, we aren’t only in your office but we are also part of the market, the one almost matters the most. Since we are tech-savvy and current, it means a service or solution provider has to keep up. Take a grocery and delivery application allowing users to shop online and have items delivered to them, users might like having a scan and pay feature. In addition, the delivery might have to be tracked for route used, items weight and so forth, and some important reports. Maybe a month to develop and rollout the scan and pay feature and two weeks for the delivery tracking. As part of maintenance we refine and optimize the app and release an update weekly. Users might ask to see how far is their groceries, ETA, as a solution provider, we will have to continuously build, integrate and deliver. This may require us to put measure in place the to reduce time to market of the solution and updates.

For immense solutions, it is not that easy. The size of the solution is directly proportional to the time to take in order to understand what exactly it is required from us. Once we have an understanding, and captured the understanding in a vast document, we then analyse and design a solution with reference to the document. So, larger document therefore a very large design, which means a very long development period. It could take over eight months before development starts. Now eight months on our current market is a very long time. The solution might even be totally irrelevant before we start developing or the might be new efficient design development suitable for solutions of this nature. Are we going to continue with the solution? Will the client be satisfied? Otherwise, the planned solution might have to be updated to accommodate new improvement, which means starting over and understand what’s expected from us and document it. Say second time around takes us five months’ tops. Do you think the market will stand still for us? See the loop? Continuing like that as a solution provider, we will diminish and run out of business.

There is hope though. There are ambitious methodologies, with goals to meet market requirements, satisfy clients, and keep the team happy. However, there is always a group in this industry that don’t welcome suites of this nature. In the following discussions we look at these ambitious methodologies.

Agile

Agile methodology is a light weight methodology compared to the waterfall approach. Its documentation, the agile manifesto, was declared in 2001. Agile approach comes as a results of seventeen enthusiasts meeting up for skiing, relaxing and talking among other things. It has various flavours such as Scrum, eXtreme programming, feature-driven and so on. However, it wasn’t known as agile until Martin Fowler referred to is as agile. This approach is for “mushy” stuff.

The manifesto outlines four values and twelve principles among other things of the approach. Agile values the team and individual over the manner of achieving and tools used to achieve — ironically this is ensured by the manner we conduct the project, for instance through the ceremonies; a viable product over an intense documentation; presence of the customer over legal and contract negotiations as the customer is involved throughout the development of the solution; flexibility to accommodate change over a fixed plan this may be because of the involvement of the customer my bring change in requirements and that change is welcomed, as a results there is an increase in customer satisfaction. Moreover, there are twelve principles, which are as follows: Customer satisfaction — this is realised through delivering a viable software continuously. Change — change in requirements is allowed at any given time, this allows the customer to remain competitive advantage in their line of business. Frequent delivery — early and continuous delivery of a viable product. Collaboration — developers work together daily with business people during the course of the project. Motivated individuals — the team is provided with the environment and support and most importantly trusted to deliver. Conversations — face-to-face communication is encouraged as an efficient means to convey a message. Progress — a working product is a measure of progress; a working prototype is demonstrated during the review ceremony. Sustainable development — the development team and the sponsors can maintain an unbroken pace throughout the development. Excellence and design — continuous attention to technical excellence and a good design is paid for enhancement in agility. Simplicity — this is simply essential. Self-organized teams — the team knows itself well enough thus a self-organised team produces the best. Retrospective — the team regularly reflect they may adjust to being for effective.

The agile approach integrated quality assurance through iterative phases (e.g. sprint) which is fairly easy to manage as opposed to managing quality in the waterfall approach with long phases. Moreover, agile approach presents us with low defect rates, as we a requirement gets analysed, implemented, tested (Unit, CIT, UAT) within a sprint instead of waiting until everything is developed. That allows promised a faster solution development and delivery while accommodating change. This approach provides visibility of the SDLC, might be too much for others. Dependencies issues, bottlenecks, politics or whatsoever type of impediments is exposed early can be resolved early.

A very significant number of practitioner believe that agile is suitable for small and medium sized projects. This may be because the team will be small or medium and agile approach doesn’t work quite well with big teams. However, agile is adopted by small, medium and large organisations. Its lack of a systematic methods, earns being noted as undisciplined. Furthermore, the agile approach focusses on the implementation aspects of the project whereas there is more to a project such as managing costs of the project. Unfortunately, agile inherits sociocultural challenges in an environment. The openness of the environment to holding agile ceremonies, team differences in communication and the teams difference with regards to welcoming change may impact the approach. Agile means no more working overtime days before the big release, however, it means working throughput the project which may lead to burnouts. Moreover, this approach is not for any project. It is not favoured on safety-critical projects. A person with the ability to bend rules in order to accommodate unprecedented circumstances is required throughout the project otherwise, this won’t feasible.

Unfortunately, agile doesn’t imply project success. First of all, this approach is not for everyone, your values must be aligned with the agile values and principles. The organisational culture has to be agile ready, or at least ready for to try it out. Unlike waterfall, agile lacks the project management processes. The One of characteristics of the waterfall approach is predictability, of which is lacking in the agile space. Requirements evolve and mature as we develop and this flexibility is awesome however we are open to scope creep. Its collaborative nature does not fit well in environments where individual accountability of code is crucial.

Let’s look at one of the flavours — scrum. As the name suggests, team, business, and developers come together to update each other, review and plan.

Scrum

Scrum process and project management framework. It is detailed and descriptive than the agile manifesto regarding specifics on how to do things. Scrum is an experimental process, harness knowledge from experiment and decisions are made based on the knowledge. It is incremental with the following behaviours: transparency — those accountable for an outcome must be fully aware of and understand what is expected of them and the understanding must be common, and inspection — frequent but not interfering with the actual work, to identify unsolicited variances. Adaptation — adjusting accordingly in order to prevent deviation from goals and objectives. It defines the team roles, activities, and ceremonies. The 5–7 team is co-located, and cross-skilled.

This behaviours are made possible by the ceremonies built in the framework. These ceremonies include Grooming, Planning, Review and Retrospective. These ceremonies are governed by the scram values which are as follows: Courage — courage to trust, protect, commit, work, and communicate. Openness — there is transparency and sharing, this build’s trust within the team. Respect — given definition of roles in the framework, team must respect each other’s role. Commitment — committing to one’s role. Focus — focus on what we as the team have committed to. These behaviours work hand in hand as shown in figure 3.

Looking at the presented roles, there’s a product owner, this is the client or someone that represent the business. The product owner presents what should be done and prioritize. Then there’s a scrum master, coaches the team. Moreover, the scrum master administrates the project, and interact with other stakeholders for the smooth running of the project. Then there’s the development team, including testers. The team weighs the stories in terms of the effort and time required to complete those items. They then negotiate with the product own on what to take into a sprint. Only the team defines how these items may be completed. The overall team work together daily; from a daily stand-up ceremony where members’ updates on what they are busy with, issues they have, and what’s next for them. It is recommended to have daily stand-ups at the same time and place and the overall time must be present. Once work has been done we can meet for a review at the end of the sprint. During review, we development team demonstrates a working prototype to the rest of the team, then rest of the team provides feedback to the development team. We later sit for a retrospective ceremony to identify what went well, what need improvement, and areas that need clarity. We wrap up the retrospective by outlining goal for the sprint, and an action plan for items that needs improvement. We meet again for a grooming ceremony, when we prioritize, add, remove or update stories. Planning 1 and 2 then follows, when we size stories and discus how to do them, then we start a sprint. To do items are tracked as stories, which describe a feature and why that feature is required. This reflects the value added. Sprint is a single iterate, about 2 weeks long. During a sprint we work on a subset of all stories. Rest of the stories are in a backlog. As the team we define what is mean by a story being done — could be that the development and code review is complete or has been moved to UAT. Stories that couldn’t be achieved in a sprint are moved back into the backlog. There are tools supporting scrum — such as Jira, which allows us to track and report on the progress. Figure 4 illustrates the scrum framework flow.

Agile isn’t one of its kind, as it is iterative, there are other iterative approaches. Furthermore, agile is a lean approach. The next discussion is based on the lean methodology. The lean methodology is all about eliminating the unnecessary and focusing on the important things.

Lean

Most critics comments on agile as “nothing new” and that is simply because agile inherits a handful from lean approach. Lean goes way back, developed and first implemented by Toyota (manufacturing automobiles) just after world war 2, under the leadership of Taiichi Ohno (1912–1990). Also, in the 1990s, aerospace industry adopted the approach. This approach is about delivering the right thing, at the right time and place on the first attempt whilst reducing muda. Muda is simply an activity that consumes resources without adding value to the overall process, in other words, waste. Muda comes in many forms, could be frequently rectified mistakes, unnecessary movements, and delays.

Lean has 5 main concepts that translate to principles. Values — this is the value of the product to the client. Only the client knows what they would like. Value stream — series of actions for delivering value to the client. Flow — value adding steps. These steps are continuous. Pull — downstream process pulls from upstream processes, eliminating inventory between processes. Perfection — kaizen process.

The lean approach brings about continuous improvements. Some of the benefits are reduced cost, development period, effective and efficient use of resources. This process builds quality in while creating knowledge (Poppendieck, 2007). The challenges of removing muda, is that muda must be identified and it isn’t always obvious.

Figure 5 is a summary of common and different aspects of lean and agile. The lean team pulls items instead of pulling them in. items aren’t pushed and rushed, this maintains quality. There’s no time-boxed sprints in lean. Lean time measure items in time not effort, time from clients request to when client receives as requested. It is the duty of the scrum master to address impediments the team faces in the agile approach whereas the entire team is responsible for that in the lean approach. Lean has a board also, the Kanban board instead of scrum broad as illustrated in figure 6.

What is completed is rolled out. This process involves integrating new product into the exiting one, building what the customer holistically wants. In a waterfall environment, you are most likely to fine people in office with wall and doors. Agile takes out the door, puts part of the team together. Lean breaks down the wall. The environment must be lean conducive. It must be easy for knowledge transfer to occur. Once the doors are out walls down, knowing and understanding the approach, we now automate where possible. Build automated tests and builds, automated deployment pipelines and forums. Lean at its peak — Devops. Agile values people over technology and processes, and lean on the other hand values work. Devops is influenced by technology such as mobile applications and cloud. It allows us to make things easier where possible, be leaner. These technologies are used to make developers work easier allowing more time for development.

DevOps

Devops is a cultural shift. It is mainly for business-orientated solutions or service providers. It extends to how we collaborate, communicate, set of technology tools and how we used them. Devops bring lean and agile concepts into one, it brings developers closer to ops. It touches on out ITIL aspects, social psychology and culture, system thing and continuous improvements as shown in figure 7.

Integration of dev and ops contradicts system engineering, where by dev and ops exist as distinct. System engineering is common in waterfall approach. DevOps practices has 5 fundamentals, which are as follows: Cross-functional team and skills — development and quality integration by agile is extended further as functional and non-functional roles are integrated. Continuous delivery — frequent updates and releases. Continuous assessment — assessment of costs and infrastructure, development and release processes. Optimum utilisation of tool set — using a tool to its full potential. Automate deployment pipeline — eliminate deployment friction.

DevOps brings about sharing of responsibilities and goal. It is no longer about developers wishing to release quickly, and operation guys trying to keep environment stable. Goals are commonly shared. Integrating dev and ops eliminates delays and loss of information during communication. It then becomes easy to deploy, not so risky anymore. No more blame game. No silos and isolation. Change is welcomed.

Through Devops, product is always ready for a release at any point throughout its cycle. It reduces the cost and uncertainty of going live. There are just enough feedback loops in this process. Enabling us to improve and keep clients happy. A happy client, and an approach such as this, means a healthy environment.

For more on DevOps, you may look at the following material. These are recommended on Devops enthusiasts.

· Getting to Yes with Yourself by William Ury

Conclusion

As much as life is too short for eject and safely remove USB/Media life is too short for a waterfall. In this day and age, we can’t afford to plan a for over a year. First, our product might expire prior delivery. Resources would be wasted. If it doesn’t expire, it must be a money machine, that could double, triple our revenue, of which it would be best to have it up as soon as possible. And the way to have it up as soon as possible is not through the waterfall approach. Culture us important, these methodologies shape the organisational culture. They indirectly specify the expected culture to them to be successfully adopted. One of these methodologies was adopted writing thing article, based on the results, which method would you say was used? The order of discussions in this paper evolve. Waterfall is an ape, lean and agile are men, DevOps is our singularity, for now. Mike drop.