For over a decade, I have been involved in software development and gone through the entire process from planning to final delivery, and I know many people who have done the same. They are free time enthusiasts and professionals, programmers, software developers, team leads, and consultants. Some got to their positions right after high school, some after college, and some moved into the IT industry from another field looking for something more modern. They work all around the world — Europe, Australia, and the United States.
But even with this diverse background, the stories I hear about failing projects are always pretty similar: Software development is hard to plan, and none has any idea how to do it. The boss doesn’t understand the amount of work. Management makes promises that we can’t keep. The deadlines make no sense at all, and there is no such thing as code quality.
Not all of these points are relevant to every story. But have you heard of a project that has none of the above? That cost exactly as much as planned, was delivered on time in high quality, and satisfied customers?
On the other hand, there is much religion out there for all these issues: TDD is supposed to increase the quality of code, Scrum removes the inefficiency of top-down planning, CI/CD makes releasing easier and offers insight into the progress to the management. And these are only some options: There are various ways of branching, versions of Scrum and Kanban, IDE-features to increase productivity, ways to communicate better and an omnipresent dislike for the evil that is outside, top-down management.
The one thing that got me thinking about all this was weirdly enough Windows 95- More precisely the question of how anybody was capable of developing it. It was released six years before the “Manifesto for Agile Software Development” (2001). Most of TDD has also only been around after the year 2000, The most widely used and arguably best IDE, Visual Studio, was released in 1997, Subversion and Git in 2000 and 2005, respectively. If you asked anyone today if it was possible to write an operating system consisting of 15 million lines of code in three years without a modern IDE, without modern SCM and with no agile team concept, the answer has to be no. I would say no myself, and I am sure so would you. And Windows is not the only example for such achievement: The Linux kernel, software used in the Space Shuttles and Missiles, the first releases of Office software, databases, the tools used to design CPUs, and the list continues.
What worries me personally is that I would consider those projects impossible even though they were actually completed. This leads me to believe that software developers are not good at judging their industry — or at least I’m not. If, as a somewhat experienced dev myself, my intuitions are off by so much, what does that say about my ability to make judgments? Or plans based on those judgments?
One recurring sentiment in all accounts of failed or struggling projects is the overwhelming pressure negatively impacting progress and quality. This pressure is generated by the management that doesn’t share the experience of the software devs and lacks understanding for scope and progress rate. The usual utterance is “Good bosses are hard to come by!” — but I have started to wonder if that is the root of the problem. Maybe we are just bad at communicating workload and the list of stuff todo. It should not be weird to have a supervisor that would not be able to perform your job. For example, many medical professionals are supervised by someone who could not replace them, yet they manage to work together and fulfill each other’s expectations. The astronauts who went to the moon and back were supervised by people, who were completely incapable of replacing them, yet they cooperated very successfully.
As a software developer myself, I know how hard it can be to make a client understand the work you do and the time it will take. However, I feel more and more, that it might be on us. You have most likely heard this story before: “Our task was to develop a prototype. Once we finished it, management thought the project was almost done. Why doesn’t management understand that there is so much more to do?”
This scenario strikes me as particularly odd now that I try to think about it another way: What do you expect them to think? If you build a system that is indistinguishable from the real thing — what do you expect a stranger to software development to think? Imagine your task was to build a spaceship to go to the moon. You would do some search online and build a paper mache replica that looked exactly like the original rocket and also delivered a simulation video of a rocket taking off. You would then meet your employer and say: “All that is missing is some of the engineering parts.” — There would be massive misunderstandings.
Rapid prototypes and clickable dummies are elaborate lies that actively hide the level of complexity and the total workload of a project, so it should not come as any surprise that they sometimes achieve that goal.
Obviously, rapid prototypes and clickable dummies are not the only problems, but they are a very interesting symptom. We expect the product we work on to be seen through the same eyes as ours when we discuss it with coding-outsiders (such as the prototypical team lead, who doesn’t understand what we do). Scrum backlogs are another such symptom. The rules for that organization pattern demand that everybody understands the core principles — but an outsider won’t. It’s a meme in the industry by now, that management doesn’t know, that velocity is a different value for every team because they estimate workload differently and use different scales. By making that information public though, we indirectly demand that knowledge.
If we say: item one is work for three people for one week and item two is much smaller, that makes the information accessible to strangers and also makes it comparable to other teams if they communicate similarly.
What I am trying to express is the idea, that the information has to come from us or we make ourselves vulnerable to misunderstanding. A Scrum team-member can translate story points to developer-days, a web designer can tell the difference between a clickable dummy and a finished product- letting an outsider do that job sets them up for failure and us along with them.
While I don’t have all the answers to this problem and some of it remains a mystery to me, keeping this problem in mind alone already helps a lot not to get frustrated and to have more productive cooperations. It also helps to understand that the other person is also trying their best and not intentionally failing our expectations. Instead, it might be on us to come up with better ways to fulfill theirs.