Developing an SDLC Process for your Organization
Where to start let us look the history of various processes
Whether you are a business leader, technology leader or operations leader, having defined processes and methodologies are critical for efficient delivery of your product or service. The purpose of a Software Development Life Cycle (SDLC) is to provide step-by-step phases that establish a model for meeting your objectives efficiently. Generally, an SDLC has been perceived as a software engineer’s domain, since the final product is a software coded product or service that can be an API, website, ad-unit, email, mobile application or a robot. An SDLC’s process should be used and should encompass all members of a cross-functional team that includes stake holders, project managers/ producers, strategists, UX, creatives, copy/ content creators, development operations, quality assurances and developers/ engineers. We will see further evidence of this need for an expressed by process experts.
First we will explore the history of such processes and methodologies to gain a grounded understanding about these ideas and models. Beginning with Waterfall, followed by Spiral and then Agile/ Scrum, we will also touch upon Six Sigma, Iterative, Incremental and Lean processes. The assessment will include key components, weaknesses, benefits and more importantly the reason why they are useful and the problems they attempt to solve.
On June 29, 1956 a presentation was given to the United States, Navy Mathematical Computing Advisory Panel, where the Waterfall process was outlined for the first time. Waterfall’s formal description was first cited in an article in 1970 by Winston W. Royce titled Managing the Development of Large Software Systems. The term Waterfall was officially used in 1976 in a paper by Bell and Thayer. The paper written by Winston Royce is the most misunderstood of them all. Royce in his 1970 paper advises against this process for software development. On the second page of his paper, he states as follows.
“I believe in this concept, but the implementation described above is risky and invites failure.”
Royce’s paper actually contains current agile principles such as design comes first, document and design, and involve the customer. Waterfall is the easiest to understand and roll out, but it invites high risk of project failure.
Typical Phases of Waterfall
- Deploy and maintain
This process is used in many industries, where in the beginning of the project the objective is defined along with cost and timing. It doesn’t always involve the customer, nor does it measures risk or change, as stated in Royce’s paper.
“Either the requirements must be modified, or a substantial change in the design is required. In effect the development process has returned to the origin and one can expect up to a 100-percent overrun in schedule and/ or costs.”
Tobias Pfeiffer wrote a fantastic blog article titled Why Waterfall was a big misunderstanding from the beginning — reading the original paper. Here are some quotes from the article.
Some may even know that Waterfall was supposedly “invented” by the paper “Managing the Development of Large Software Systems” by Dr. Winston W. Royce in 1970. But have you ever read it?
If I told you that it is a paper identifying this pattern (without naming it Waterfall) and describing its deficiencies and proposing enhancements?
Risk assessment being a weakness of Waterfall, the Spiral Model addresses this concern. In Barry Boehm’s 1986 paper A Spiral Model of Software Development and Enhancement, the author methodically distinguishes between a process model and software, as well as makes other key points. His notion of a process model supports the idea that an SDLC needs to be extended to the entire group not just developers. Boehm makes other points similar to Royce about Waterfall, as well as other insights.
Document-driven standards have pushed many projects to write elaborate specifications of poorly understood user interfaces and decision support functions,
Recognition of the feedback loops between stages, and a guideline to confine the feedback loops to successive stages to minimize the expensive rework involved in feedback across many stages.
Boehm also authored Software Engineering Economics in 1981 and The Incremental Commitment Spiral Model: Principles and Practices for Successful Systems and Software in 2014. The titles of his books themselves reveal the common themes of cost management, iterative phases and risk analysis. This process is also described as a process generator, where after each cycle the team decides the next steps for a project.
Spiral model of the software process.
Typical Phases of Spiral Model
- Determine objectives, alternatives, and constraints
- Evaluate alternatives, identify and resolve risks
- Develop, verify next level product
- Plan next phases
The Spiral Model is broken into mini waterfall cycles. The first few cycles put more emphasis on planning and risk resolution. In each cycle after the initial risk analysis a functioning prototype is developed and the entire team reviews the value of the product from their functional expertise. To manage the cost per cycle as the market or customer value is measured, additional process rigor is put in place to ensure quality and production readiness. What is also important is that there is a feedback loop between stages of development, the wider involvement of other roles and user-interface considerations.
Agile/ Scrum Framework
Apart from Waterfall, Agile is very familiar to everyone, both as a buzz word and software development methodology. Like Waterfall it has been around for just as long. Gerald M. Weinberg, was quoted in Iterative and Incremental Development: A Brief History by Larman, Craig; Basili, Victor R (June 2003),
“We were doing incremental development as early as 1957.”
Later, from the early 1970’s up to late 1990’s, an industry debate raged regarding the need for lightweight development methodologies. This rose from the frustration against complex, monolithic and oppressive processes.
In February 2002, Ken Schwaber, Ken; Beedle, Mike wrote Agile Software Development with Scrum. In a talk at Google, Schwaber gave my favorite description of Scrum, stating that Scrum isn’t a methodology, but a baby process that is like the game of chess, where the moves are simple, but you put them together to device very complex strategies.
Schwaber’s assertion that Scrum isn’t a methodology, but a process aligns with Boehm’s Spiral Model, where you have to separate the process from methodology because they are different. One is imperative and the other is declarative. We will dive deeper into this.
Typical Phases of Agile/ Scrum Method
- User story development or product requirements from a user perspective
- Sprint planning, breaking workload into reasonable iterations and estimating
- Work begins, based on the current sprints deliverables
- Sprint review, work is evaluated
- Next sprint planned and the cycle begins
- Project retrospective, the team gathers to see how they can improve and
- Adapt new experiences learned for future projects
Agile in it’s simplest form is a to-do list, much like doing chores and preparing thanksgivings day dinner. Its objective is to quickly provide value to customers, welcome change and allow the development team to focus and work efficiently. It also provides transparency and an adaptive feedback loop for learning and improving. In the same talk at Google, Schwaber states that, “Scrum is a collection of best class ideas”, which actually includes Waterfall and other processes.
It’s also a quality control system similar to Six Sigma. Jack Welch the CEO of GE made Six-Sigma the central focus of his business strategy in 1995. It was broadly applied to all aspects of the GE organization, including NBC, a media and entertainment company.
Creating Your Own SDLC
With a brief historical overview and feature highlighting in Part 1 of this article, now let’s look at how to go about SDLC creation.
- Plagiarize, find what works and use it, don’t reinvent the wheel. Some of these processes have evolved over 120 years, for example, the core concepts behind Lean started with Fredrick Taylor in 1890 with his principles of Time Study and standardization of work. If you take this highly recommended route, it’s important that you understand the process, get trained and have active senior leadership support.
- The right people, if you feel you need processes in place to make poor performers do better, then you have a staff issue not a process issue, unless you don’t have the right process and everyone is tripping over themselves.
- Know the difference between a process and methodology, similar to imperative and declarative programing languages. Imperative languages mean that you have to define how to do something and declarative languages are more about what needs to be done. Thus, process is declarative and methodology is imperative. Barry Boehm’s Spiral Model makes a great point that, “Thus, a process model addresses the following software project questions:
- (1) What shall we do next?
- (2) How long shall we continue to do it’?”
- Methodologies are important, and should be documented into policy and instruction guides and continually maintained.
- Waterfall can’t be a standalone process, nor a parent process to other processes like Agile, Scrum or Six Sigma. Waterfall must be a child process of other more iterative and introspective processes, unless your organization deliverables have very short cycles and are very simple.
- Processes can be combined, for example Six Sigma tools inside an Agile framework. where you can use the two Six Sigma quality tools DMAIC and DMADV as a part of Agile.
- Leadership needs to play a key role, in the same way Jack Welch drove Six Sigma. In the book by Robert Slater Jack Welch and the GE Way, it took Welch a number of years to get there. It started with a touchy feely quality concept and over time went on to become clearly defined processes and methodologies with mathematical formulae to measure efficiency.
In the book Good to Great: Why Some Companies Make the Leap…and Others Don’t by Jim Collins, one of his key principles of great organizations is first finding the right people.
“Great vision without great people is irrelevant.”
Your people are your best luck, as he later describes in the last chapter of his last book Great by Choice: Uncertainty, Chaos, and Luck — Why Some Thrive Despite Them All, with co-author Morten Hansen. Your processes and methodologies are there to support the good people you have. As expressed in book The Art of Scalability: Scalable Web Architecture, Processes, and Organizations for the Modern Enterprise, Edition 2 by Michael T. Fisher and Martin L. Abbott,
…virtuous cycle of people and process scalability to support better, faster, and more scalable technology decisions…
This idea is supported by the demand for lightweight and micro-process to remove bureaucracy and waste.
Next step for you is to dive deeper into the various processes that are out there. Do your own research, comparison, and analysis. Then select the process or combined processes, that your leadership and culture can get behind. You know you have selected the right process or processes, when the process quickly exposes a problem and you are faced with making a decision. Conversely, you have the wrong process or no process, when you think everything is going fine and you don’t see the hidden problems.