From Prima Donna to Agile in 20 years
“Then you better start swimmin’
Or you’ll sink like a stone
For the times they are a-changin’.”
Back in the 80's, I managed the software development department for a successful one hit product company in Dallas. My boss, the CEO, used to tell me: “Of all the executives in this company, you are the only one I cannot manage because I don’t understand what you do. I just have to go with what you tell me.” I felt powerful. In the R&D department, we created the product that made the company successful, and I was the undisputed master of that domain. I felt like a prima donna and transmitted that sentiment to my people. “They actually think they are our equals,” we used say jokingly whenever anyone outside of our department dared question our judgment.
Coming from the programming trenches myself; I was a big advocate of this special status for software programmers. I started out as a C programmer, and I had a good amount of programmer’s credit to my name, including multiple all-nighters, weekends and holidays in the office, and a record three consecutive days and nights just before a ship day. We had PC Magazine “Product of the Year” awards, PC World awards, and other accolades to back it up. “You can’t mess with these guys,” I used to tell my fellow executives. “They are the heart of our value chain, and they need to be creative and efficient. For that, they need their space, and we must give it to them.”
The feeling lasted for a while — perhaps into the early 90's or so. Time to market seemed to be the big issue. We had to quickly write software and deploy it. Deadlines were negotiated — if other departments wanted the ship date to go their way, they had better be nice to us and thankful for our great programming minds. Never mind the fact that software never shipped on time — and if it came close to the target date, it was buggy. That was soon forgotten. We would put bugs out in the hands of customers and then feel like heroes when we fixed them rapidly. Even worse, marketing, sales, customer service, tech support, the customers — everyone — thought we were heroes.
Then, it happened. Competition and the commoditization of writing code forced us to be more quality conscious. It was not until then, and I admit it with shame, that I fully understood our mission: to write business software that helped customers to become better companies. “We are not here to show off how smart we are,” I thought. “We are here simply to help the customer.”
Now there was a new sheriff in town: the Quality Assurance Analyst. The last vigilant barrier over which that capital sin, bugs in published software, could not go through. It was a big transition. I was the VP of R&D and I did not respect QA as much, even though they reported to me. “This is ridiculous,” I used to say. “Now, it takes longer to test the program than it takes to code it.” But it was the nighties, for the times they were changing.
Competition made the industry more quality conscious. We developed metrics to measure quality, and QA was the big savior. A year later, and it was all about Quality Assurance.
The Process was still broken. By empowering our QA department we had built a solid check point right before we put software in the hands of the users. But the process was still flawed. The rest of the journey was the same. From idea to design, design to coding, and coding to mastering was still the old waterfall chaos where it was not clear what fell through from one phase to the other. The relationship between programming and QA became an adverse one. QA Analysts were the custodians of the gates where code had to be carefully scrutinized prior to getting through. In the end it was like the Tijuana-San Diego border at that time, the lines were huge and there was not any guarantee that a few troublemakers would not make it through.
But Times they were changing… and along came Agile. Some time in the first half of the 2000’s we enthusiastically adopted the Agile Scrum project management methodology. It would take as many paragraphs to briefly outline the many virtues of Scrum. For now it suffices to say that at the heart of it, it puts a multi-disciplinary team together in every facet of the development of the software. Programmers, QA Analysts, documentation writers, and others work together with a Product Visionary, the Product Owner, to interpret the needs of the stakeholders and to decide when and how it will be developed, tested, documented, deployed, and implemented.
Scrum is a highly interactive project management methodology, which embraces teamwork, flexibility to adapt to changes, and short cycles for deliverables. This is not an article on Scrum but more a history of an evolution. Perhaps shortly I will follow up with an article that describes Scrum in more detail. For now it suffices to say that Scrum was definitively a game changer, but do not stand still as “for the Times they are changing…”