Revisiting Agile, Part 1: The Origins of Agile
The world is digital, and this is not news. “Digital marketing” is “marketing.” Likewise with “digital strategy” and “digital product development.” Whether you’re a CMO, a copywriter, a graphic designer, an accountant, or an illustrator, you are trafficking in digital artifacts. Just about anything analog — even that new vinyl record you bought — had a digital precursor (in Pro Tools, perhaps) and made its way to you thanks to a digital supply chain.
Whether we say it out loud or not, we all know our jobs are digital. If we need to deliver something, digital things need to happen. And if we deliver our work in a digital medium, there are bound to be technical requirements. A marketing director with a background in copywriting is responsible for things like SEO, page speed, and accessibility. A studio executive with a film background is responsible for bitrates, aspect ratios, encoding, and digital rights management.
Technology has enabled amazing things in our lifetimes, but it has also created new professional burdens that were unimaginable twenty years ago. Working culture has not kept pace with these new, rapidly changing demands. Even in the purely digital domains of software and the web — those that led the way into the 21st century — we need a new philosophy of working.
How it was
When the internet was young — when digital technology was arcane, mysterious, and decidedly unsexy — those of us old enough to remember could take refuge in our analog world. We made phone calls by keying in numbers we had memorized, took notes on paper, and left the digital magic to The Programmers (creepy music plays in the background).
The Programmers — both revered and feared — held the keys to a magic which was opaque to everyone else. They had a reputation for speaking in strange tongues, avoiding social gatherings, and keeping out of the sun.
Afraid as we were of The Programmers, we did want more of their digital magic. Although early internet technology arose from military and academic pursuits, consumer demand was never far behind. We wanted easier, more convenient ways to get our news, chat with each other, buy stuff, share photos, listen to music, keep in touch with old classmates, and even watch TV.
Enter The Business People. The Business People, recognizing the great commercial opportunities presented by digital technology, enlisted The Programmers to make some of this stuff people wanted.
The Programmers, being very literal and precise in nature, needed very detailed descriptions of anything they were to build. With a detailed description in hand, they could estimate the work and tell The Business People how much it was going to cost.
Having provided a very detailed description of The Thing to be built, and having approved the budgetary demands of The Programmers, The Business People would wait. They’d wait for months — or even longer — for The Programmers to do their magic and deliver The Thing.
As The Thing reached completion, The Programmers would invite The Business People to review what was built. The Business People would behold The Thing and say, “You left out all of these common-sense features! You couldn’t have thought we wanted X without Y!” To which The Programmers would reply, “We have satisfied every requirement you provided. If you would like anything else, we shall need new requirements and a new budget.”
Change order hell
The Programmers could work for a third party or for the same company as The Business People; the outcome was usually the same: the budget had been blown, and The Business People did not have The Thing to sell to consumers. Reluctantly, The Business People would write up new, more detailed requirements as the basis for a change order (an amendment to the original contract), a new contract, or a new budget plan, providing more money and time for The Programmers to continue working.
The requirements Olympics
The Business People thought, if the requirements were perfect, they could pull off the next project without Change Order Hell. So, they hired business analysts and systems analysts to compose exhaustive requirements, and they reviewed those requirements (usually dozens or hundreds of printed pages) with great care before handing them over to The Programmers.
Every once in a while, it actually worked. If you had a brilliant business analyst and a brilliant systems analyst, and the giant spreadsheets were just *chef’s kiss*…and you had an experienced development team and a killer quality assurance team…and business conditions didn’t change significantly in the 18 months it took for all of this to happen…
The Agile Manifesto
Several processes were put forward and implemented across the industry by The Programmers, each of which sought — among other goals — to eliminate or mitigate Change Order Hell. While the processes differed, they shared certain themes: more frequent communication between The Business People and The Programmers, more frequent reviews of work in progress, and mechanisms for updating requirements without needing to rewrite contracts. The Programmers were on to something. These new processes, including Unified Process, extreme programming (XP), and scrum, gained traction and notoriety — because they really helped.
In 2001, some of The Programmers, including the architects of the aforementioned processes, got together to ask, “What do these new processes have in common that makes them successful? Are there guiding principles to successful software development?”
The “Agile Manifesto,” as it is often called, had an immediate and lasting impact. Associated processes, especially scrum, became synonymous with agile software development. Organizations the world over (although surely not all of them) got out of Change Order Hell and embarked on a more iterative, collaborative way of delivering digital products.
Is the Agile Manifesto still relevant?
Not everything about the Agile Manifesto has aged well. To the authors, there were two kinds of people: “the customer” and software engineers. This customer/engineer dichotomy — aka the Business People/Programmer dichotomy — was a useful approximation in 2001. But today, it is completely invalid.
