Introducing the Core Process for Developing Software
(note: this is a rewrite of this article)
Whether its Waterfall, an Agile methodology (such as Scrum), a custom-defined process specific to your organization, or one of the many other published software development methodologies (RUP, Crystal Clear, etc.), software developers follow some kind of process to build their software. Even for the student working on a homework assignment to write a b-tree, there is still some process being followed.
What if there was a single, well-defined Core Process that ALL of the existing software development methodologies (and even the one followed by that student doing their b-tree homework assignment) followed? What if you had a fundamental understanding of what made up these methodologies? What if you were asked to define the methodology that your organization was to follow to develop software?
This article is going to attempt to define and introduce the Core Process, a process that all software development methodologies are theoretically built upon (that is, they all incorporate this process in some form or fashion). In fact, the Core Process can be used as the basis for developing any and all methodologies. It can also be used as the basis for improving any existing methodology.
That’s a lot of hand-waving about a very large industry with a lot of very experienced individuals with very strong opinions on how to build software. But for all of the different methodologies invented, none of them address the foundation upon which all methodologies are defined and upon which new ones can be created and improved. So, here’s the introduction of the Core Process: the foundation of software development as a process.
Origins of the Core Process
More than a decade ago, I asked myself an interesting question: what is both necessary and sufficient to build software. That is, what must one do to build software (from a process-perspective) without doing anything that isn’t absolutely necessary (that is, it’s sufficient)?
As I approached the question, one way to help me understand whether a particular step was really necessary was to ask what would happen if I didn’t do it. It was that question that opened my eyes to the real answer. Why? Because my natural inclination was to include some form of testing in the process. However, “what would happen if I didn’t do that?” merely gave me buggy software, but it still provided me with software.
Take any methodology and take away every practice it requires (or recommends) and only add back in those things that are absolutely necessary to get something (regardless of how good or bad it is).
This is what I got…
Requirements
You must have requirements. That is, you have to have some idea of what you’re building. I like to use the example of the million monkeys who are typing on typewriters for a million years trying to get the works of Shakespeare. If you left off wanting to get the works of Shakespeare, you’d merely have a million monkeys typing on a million typewriters for a million years…and a lot of spent paper and ink, and no idea what was produced.
Another way to look at this is the reference to Alice and the Cheshire Cat in Alice’s Adventures in Wonderland. To quote:
“Alice asked the Cheshire Cat, who was sitting in a tree, “What road do I take?”
The cat asked, “Where do you want to go?”
“I don’t know,” Alice answered.
“Then,” said the cat, “it really doesn’t matter, does it?”
Without requirements, what are you building? And, does it matter?
Building
On the surface, it may seem obvious that one of the steps is to actually build the software. However, it raised some interesting questions. For example, “design” isn’t one of the steps. Why not? In fact, Jack Reeves argued that coding IS design. What about some software systems that are more configuration than code? What about no-code software development?
I use the generic term “build” to capture any and all of the different approaches used to build software, whether it’s a more traditional programming language, a no-code platform, configuration of an existing off-the-shelf system, or whatever. Build in this context means that work is performed to meet the requirements specified in the previous step. It is implementing the requirements.
Publish
Publish is another generic term that refers to delivering, using, selling, installing or otherwise making use of the software that was built.
It may seem obvious, but it can be argued that millions of dollars have been spent on “developing software” that never saw the light of day. These cancelled projects can be argued to have followed a process without “publishing” it. My argument is that they probably didn’t produce software, and the instant you prove to me that they did, I can state that they were “published” in some way.
It’s similar to the argument of the tree that fell in the forest where no one was there to hear it. Did it make a sound? No, and the instant you produce ANY evidence that it did, then it did; otherwise, it didn’t. This could even be that student working on that b-tree assignment showing their code to the Teacher’s Assistant to get a grade: it was published.
You Forgot Testing
Almost without fail, the biggest issue I hear when I introduce this Core Process is that I forgot testing, that you can’t build software without some form of testing.
Years ago, I had created a website (now defunct) that was an attempt to document my thoughts and build on top of the Core Process. One day, I had an associate storm into my office and throw down a printed copy of one of the pages from the website. He exclaimed that he couldn’t believe that I didn’t believe in testing.
Of course, he hadn’t bothered to keep reading where I appropriately identified where testing fit and to actually showcase how fundamentally important it was. So, if you’ve waited this long, here’s where it comes in.
Any and all practices, steps, tools, improvements, etc., that are added to the Core Process are to improve quality. In other words, testing isn’t a part of the Core Process; it is the overarching theme that makes the process useful.
What is Quality?
I’m not going to define quality here, but what makes understanding the Core Process so powerful is the realization that everything added to it is to address quality. EVERYTHING. If you have an additional step or practice that doesn’t improve the quality of the software your building, then don’t do it, because you fundamentally don’t need to.
Just remember that every practice, step, tool, etc. that you add to the Core Process comes with some form of cost, whether it’s time, complexity, money, or whatever. This cost is the yin to quality’s yang.
Though I’m not going to explicitly define quality, it should be clear that it isn’t just the amount of defects; it may refer to may other aspects of developing software, such as time, cost, customer satisfaction, developer satisfaction, revenue, etc.
A Word About Process Improvement
The Core Process is the foundation upon which all methodologies are built. Adding practices, processes, tools, etc. can improve the quality of what’s produced. Understanding this foundation can be used to analyze existing practices to ensure they are providing the quality you want for the cost they require.
So, process improvement is merely adding (or removing) practices that are “worth it” (from a cost perspective) or not worth it (and should be stopped). That seems simple on the surface, but there’s obviously a lot of subtlety (which is out-of-scope of this article).
Conclusion
The Core Process, of itself, isn’t going to change how you develop software. Understanding the process and the implications in regards to quality can help you think better about the process you’re following (few ever get to choose which process they follow). In addition, it can help you become more active within your team and organization to identify areas that can change (by adding or removing). Just as understanding basic data structures can help us improve our code, the Core Process can help us understand the process we follow to build software.
