Agile is not an excuse to start without a plan
A wild customer appears. He asks:
I would like to have an app for this with a web application for that. Can you make that work?
So I say:
Of course I can make that work! I’m an engineer after all. It will take about six months in total, and I can give you a rough estimate of costs. But I need two weeks to do some further analysis and engineer more detailed requirements, after which I can give you a more complete overview of what it will take and what you will get.
And the response:
Can’t we just do it agile?
To which I reply:
You keep using that word. I don’t think it means what you think it means.
And then I get blank stares because using The Princess Bride memes in real life is generally a bad idea.
Agile versus planning
As humans, and software engineers in particular, we have a tendency to take good ideas and misinterpret them. REST is an incredibly misunderstood and misused term, but so are the blockchain, NoSQL databases, communism, capitalism, Big Data, and, well, most other things.
Agile has, to many, become synonymous with starting without a plan. When you do agile, you should just start coding, fail early and often (or all the time), and keep going into the direction that looks best at any given moment. You don’t take any time up-front for analysis and requirements to set a general course, you just start on whatever seems nicest to work on.
A metaphor: I want to sail around the world. Let’s do it agile! Just jump in the boat, fail early, fail often, and blub blub.
Now wait just a gosh darn second! Did you even read the agile manifesto? Didn’t it say we shouldn’t plan, or something like that?
Let’s revisit the relevant part of the agile manifesto:
Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan
You’re probably thinking of the last line. Read it carefully though. In agile, you should be flexible so you can respond to change. But to be able to respond to change, you first need a baseline that can change.
That’s what we call a plan. And simply put, it’s a bad idea to start anything of sufficient complexity without a plan.
Planning and agile
Software development is complex. If you manage to complete any complex task successfully it’s usually thanks to proper planning, discipline and perseverance. Without those things, it’s up to sheer dumb luck. We accept this fact in most fields, but seem to have trouble with it in software engineering.
Let’s revisit our sailing metaphor. You get on board and set off in a direction that looks nice. The definition of what looks “nice” changes over time; perhaps you start by sailing away from clouds, while later you would prefer to sail towards the sun. You respond fluently by evaluating and, if necessary, changing your direction every two weeks. You keep doing this until either your supplies run out (i.e. your startup funding is gone) or you find land.
Your early choices are the biggest predictors of whether you will survive. How much food do you take? Do you have enough supplies to get back? Will you keep land in sight, or will you keep going out into the open ocean? But by sailing agile without a plan, you spend exactly as much time on these initial choices as on the later choices. And at no point during the sailing trip do you know whether you will make it, or even how far along you are.
But of course, given that you don’t know, or rather never defined what “it” is that you want to make, it doesn’t really matter where you end up anyway, right?
One of the complexities in software development comes from the enormous number of choices you have to make, with especially those early choices (when engineering requirements, selecting technologies and creating an architecture) having a profound effect throughout the entire project. A small mistake can result in a large deviation of your intended destination, and without any signposts along the way you have no idea how you’re doing.
So that’s why should take a GPS device with you when you go sailing. And a map of currents, and a compass, and a lot of other navigation equipment. And you decide ona general destination before taking off. You are of course always free to change this destination, but you do decide on one before leaving.
In a software development project, you create a project plan and an architecture. You set milestones, you create the essential requirements to a level of detail that is required to plan and estimate with, and you create a rough plan and estimates. That is the navigation equipment you should take with you when you develop a software system.
This however of course doesn’t change the fact that you are happy to respond to any change if there is an opportunity to bring more value. You embrace these opportunities, as that is the core of working agile. If you see a beautiful beach or a nice shore town, of course you’ll check it out. The difference is that you now have the tools to measure the impact of a proposed change, and to actually determine how the value of your expected end result will change (both positive and negative).
Should we then just go back to waterfall?
This is the response I typically get at this point. I think it’s important to address, because I feel that a lot of misunderstandings in agile come from contrasting agile methods with the waterfall method. An us vs them team sports mentality kicks in, and we start to avoid and even despise everything that characterises the waterfall method, because it’s the “bad guy” that is not on our “Agile-team”.
Waterfall, however, is not the opposite of agile, in the same way that a cruise ship isn’t the opposite of a sailing boat. A sailing boat is much more flexible than a cruise ship, but just because the cruise ship stays afloat shouldn’t make you want to sink the sailing boat. Consequently, if waterfall tells you to plan ahead extensively, that doesn’t mean that you shouldn’t plan at all using agile.
Regardless, what I’m describing isn’t waterfall. It’s proper software project management and development. It’s professionalism, not dogmatism.
Skipping up-front work is bad for the client
For me, this is the big one. Sure the client says he wants to work agile and just wants to get results fast, but he also wants to know what he’s going to get.
In an ideal world, clients would like to go to the software supermarket and grab a certain piece of software from the shelf, for a fixed price that does exactly what they wanted all along. The exact opposite of that is what unguided agile gives you: a constantly varying investment for an unknown amount of time to get some piece of software, for which there are no guarantees that it will do what they want it to do.
Starting with planning however is understandably unconfortable for them. They expected software and code, but now they’re paying for documents and ideas and brainstorming sessions. Where’s the code?!
But none of them would start building their house the same way we start building their code, even though the investment is often comparable. In my experience, clients are definitely willing to listen to reason, and there are always ways to give them something tangible they can show off with (like a prototype) while simultaneously planning and brainstorming.
Wrapping up
Don’t get me wrong: Agile is awesome. I just don’t believe that every sprint should be seen in isolation. It’s always part of something greater, and that “something greater” often doesn’t get nearly as much attention as it should. Scrum for example hardly addresses it, if at all. But that doesn’t mean it’s not there, or it shouldn’t be addressed.
Anecdote time. For a recent project, I got stuck with an unsupported and buggy frontend framework because a team started “agile”, wanted to use the latest toolchain and this one was the “fastest” framework out there at the time. Turns out the client didn’t care about performance and just wanted a back-end application that processes spreadsheets. Nobody thought to ask.
The author of this post doesn’t know the first thing about sailing. He did read Life of Pi though, wasn’t there a sail in that at some point? At least a boat and a tarp, right?
