Back to The Future of Agile Software Development
This essay is inspired by Ken Schwaber’s unSAFe at any speed blog post. In brief, Ken cautions against returning to the RUP practices disguised as an agile framework and urges his readers to keep their commitment to Agile Manifesto.
While I’m very much with Ken on the subject of heuristics, thinking for yourself, and sticking to the principles of agile, there are some deeper things to that than merely management vs. software developers, or RUP vs. agile. Make yourself comfortable, as this is going to be an interesting 7+ min read, or whatever more time you might need to contemplate it.
The Origins of RUP and waterfall (’80s — ‘90s)
Let’s look back at RUP, waterfall, and other such unified processes, you name them. Where are their roots and when did they actually make sense? As far back as in the ‘80s-90s of the last century, maybe even in the ’70s. The unified processes are the legacy of industrial and agricultural age production methods from the earlier times of the 20th century. Back then, in the ‘80s-90s, it was natural to think of software development as a yet another field of production, not any different from hardware or commodities. Besides — and it’s an important thing to note — the pace of changes in the industrial and agricultural environment has always been slow. No changes for 50+, or even 100+ years in some cases.
First, everyone thought that software production is going to be like that as well. No changes, hence no changes in the market/production environment, hence no problem-solving skills required. Well, if the problems related to an unchanged routine activity have been solved once and for all, all you need to do is follow some rules and commandments. No new problems -> no changes -> rigid processes. With software development, it all appeared to go along in the same fashion, and on, with the boom of software development outsourcing in the late 90’s — the early 00’s, too. I used to work at an outsourcing company, by the way, and I remember how we’d court the prospective clients with the neat RUP diagrams, as we looked to assure them that their project was going to be done on time and within budget.
Next, with no changes in the environment, rigid processes are supposed to work fine. Well, fine it was until some moment. It appears that by 2001 (maybe it was related to the notorious Y2K problem, along with the others) there came an understanding to the thinking people in the software industry that something was wrong. Rigid boilerplate processes did not seem to be working to create viable software in the changing environment. This critical mass of changes found a small outlet — just as the volcanic lava first appears only as a tiny stream — in the 2001 agile manifesto, and Ken Schwaber was one of those who signed it.
I see Agile Manifesto as the soul’s cry of creative software engineers who wanted to say it out loud, that they are aware of this problem, they claim this awareness to the whole world, affirming that they’re not “code monkeys” but thinking people who have their solution to this problem in mind. They wanted to stand out and share their beliefs about better ways of developing software. They just couldn’t live with that awareness and not share it anymore. I haven’t studied the biographies of the agile manifesto founding fathers but for some reason I’m sure they’d had enough of enterprise softdev companies running in the Procrustean bed of the 20th century industrial ways.
The Agile Manifesto and what came next (2001 — ?)
After 2001, nearly two decades have been filled with the buzz about agile adoption, fading or getting louder at times (and by now, many are finding themselves in the post-agile era). Likely, the agile manifesto won its first ardent followers among bootstrapped softdev start-ups and some small web service shops. Or, among the software engineers who simply felt they can’t do their work well within the old paradigm. The agile manifesto appealed to anyone who was not a mere “human resource”, not a cog in the rigid process but a thinking person who wanted to finally be able to solve real problems, as opposed to feeling stuck with the rigidity. And, that’s what the essence of this “early-on” agile was. Agile problem-solving in the fast changing environment.
Let’s flip the phenomenon called “agile” and look at the other side of the coin. As agile methodology got its entourage of followers, agile coaches, trainers, various certification programs, it started showing some signs of rust. Like an idol worshiped just out of tradition, because everyone seems to have forgotten how it appeared originally, in the first place. Now, consider this. The industrial-style companies of the late 90’s are still there. They still have their problems. They’re still the same as in the late 90’s, or even late 80’s. They have budgets to spend and outsourcing teams to run. But since RUP and waterfall have publicly been coined as malpractices, these enterprise companies must have jumped for joy as they saw someone approaching them with the stuff called SAFe (Scaled Agile Framework). As far as I can see, SAFe has the same essence as RUP (correct me, if I’m wrong) but now it wears a nice glittering gift wrap with the big word “AGILE” on it. The terms are different from RUP, though. It’s called SAFe, but the essence is the same: it’s a prescriptive framework for industrial/agricultural style production in software development.
Such companies, indeed, must be happy to open their checkbooks to something like that, as Ken says in his blog. SAFe is what they need, and they will run fine with it, as long as the global market and business environment lets them live this way, keeping the patterns of production where they don’t need problem-solving skills in software development as the ultimate prerequisite for running their businesses.
Now, speaking of the global market environment. Of course, the distinction is not black-and-white. There can be companies who are enterprise, who are changing, transitioning, transforming their businesses, etc. Actually, SAFe might be able to help some companies. Of course, there can be a hybrid of enterprise thinking and creative thinking (the Japanese look like the most obvious example of that one to me).
“Agile” as buzzword and “agile” as continuous problem-solving
I want to highlight the distinction that there’s agile as “agile, the buzzword”, the silver bullet that many heard of, the idol that is supposed to heal all the ulcers if you worship it. Usually, people resort to this kind of agile if they:
a) don’t have time to dig deep, study, and learn from their own experience. Well, it’s one of the most common misconceptions of the humankind: a belief that your particular problem, in your particular personal/business/market context has a ready-made solution out-of-the-box, coming from some 3rd party. The ready-made answers can only come to a purely technical problem, such as the sequence of steps to assemble a wardrobe purchased from IKEA. I’m amazed at the questions people ask at some online forums, as they believe that a problem in collaboration, or production, or in a process peculiar to their organization can be resolved by someone’s quick online answer. There’s no prescription for such problems, period. Only the experience-based flow and thinking, iterations and corrections in response to the feedback from your particular business/production/market environment are viable.
b) if they’re the folks who miss RUP. They still need something like that. Budgets to spend, reports to write as they operate in the silos, undisturbed by the changes for one reason or another (so far) since the late 90’s or since even earlier.
The 2nd kind of agile — and I don’t want it to be a rusty buzzword — is about problem-solving out-in-the-trenches. Take a living organization that operates in a very fast-changing business/market/production environment. Well, I don’t think that such organizations can afford thoughtless copy-pasting of abstract prescriptions. By the way, here’s an interesting brief summary of an agile conference. In short, the best thing about this conference, according to the author, was mingling with the folks who have the same problems, but the conference gave no actual answers. I once wrote a post called Agile Conferences: Look To No Epiphany which went along the same lines.
The logical conclusion? People. People. People.
If you do not resonate with copy-pasting prescriptions for your organization, then PEOPLE are your biggest asset. That’s the real agility. Only people can be agile. Not a process, in the very true sense. Agile people can tweak a process as they need to solve the problems that appear in the fast-paced environment where they operate. The people who make one team together and are able to solve problems continuously, because a changing environment always entails new and new problems. In software development terminology we have the term called “continuous delivery”, and my version of the buzzword definition would be:
Agile is continuous problem-solving.
Now, what do you think the future will bring? More of those 20th century industrial/agricultural corporations? Or more of the indie companies with their universally knowledgeable people, who are, well, agile and alert because otherwise they won’t survive in the changing environment? This is an interesting subject in itself, and it goes far beyond this essay.
People are people. They have their problems. They live. Life changes. Agility is a way to survive, live, enjoy those moments, and create some nice software stuff that makes the lives of other people easier and/or nicer.
So, that’s where the answer to the question “ What will become of agile in the next few years?” is. It’s in the dynamics of the global economic/business market, and in what this market needs more of: agility as agility, or industrial enterprises with their rigid homeostasis untouched by the changes? With start-ups, quite another kind of homeostasis has been observed lately, but my today’s essay is not about that. Why and how the rigid homeostases of all kinds might stay untouchable would make a subject of another essay :)
Related:
This story is based on my earlier article.
