Why your Agile project “failed”

The lessons I learnt thus far, working on Agile projects…

We’ve all been a part of an “agile” project that “failed”. Failed could mean it was either late, didn’t deliver all the features people (management) expected it to deliver, or wasn’t delivered at all.

However, Agile is often hailed as the be-all, end-all solution to all the problems mentioned above. So how is it possible that projects following various agile methodologies still fail? This post is a list of some of my views on common problems, along with what I think are the steps that should be taken in a company to mitigate them.

Missing Deadlines

Planning is hard. It’s really hard. It’s hard because it often comes tied in with estimation. Estimation, though, has been referred to as a black art, and books have been written about it.

Lesson #1: Never set a deadline without pre-conditions

I learnt this the hard way. The problem is, when you estimate a deadline, you make assumptions. For example, you know you need to hire 2 more people, so you assume you’ll have them on-board. You estimate the velocity of all the people (+2). Then, hiring them fails. Whatever the reason is, you have -2 people for that estimation. The deadline comes, and makes that wonderful whooshing sound as it passes. Management is pissed at the team, saying that you set the deadline. So, make sure you set deadlines along with pre-conditions: “With 5 people, we should have features A, B, C and D, as they were drafted, by around X in the best case. If we hire 2 more we might increase the velocity and roughly gain Y.”

This leads us to:

Lesson #2: Throwing bodies at the problem is not always a solution.

A friend recently reminded me of this fact, but you can’t pay 9 women to deliver a baby in 1 month. Some things just take time. So, yes, sometimes you need to hire more people to get things done, but when you do hire, make sure you hire the right profile for the job. If your developers are working 12 hours a day, you might need to add a good developer to the team, not a manager.

Lesson #3: Always have a Project Sponsor & Management Buy-in

If the management team does not believe in the Agile way/methodology, you will never get far. Additionally, the project needs to have a sponsor. Ideally, someone from the higher-ups (e.g. ownership of the company), who is able to stand behind it, if the shit hits the fan. Everyone must know that agile is not a magic bullet. There are still things that need to happen.

A successful project requires both project governance and project management. Scrum defines how to do the tactical project management. On top of that a strategic project governance is needed.

There’s an interesting post about this topic, here. I suggest you read it.

Lesson #4: Estimations of drafts need to be ballpark estimates

The big issue, in my eyes, is that management usually wants or expects specific deadlines, even before they actually know anything about the project. It is incredibly hard to give a timeline, let alone a due date, before you actually know what you are building. Agile doesn’t change this.

There is really no way to avoid estimation. But the problem, in my opinion, is the quality of the estimate. I would start estimating by saying: The way the feature looks now, we could have it ready in 1–2 man months.

The sad reality is that management often expects us, developers, or tech leads, to give exact estimates early in the game. For example: “We want to build a tool for invoicing customers. When can we have it ready?” Does this sound familiar?

Granted, the problem is, that there are other services in the company that require a more detailed plan/roadmap. For example, marketing should be able to plan activities. That’s often the problem — discrepancy. Developers don’t want to give estimates, but the company needs them.

All in all, though, here is what I would do differently:

1. Start with a high-level product roadmap. I have recently come across a tool called Aha. I have not yet played with it, and am planning to, soon. But it seems perfect for what I want. It helps PMs define what the product needs to know, but very inaccurately.
2. When an idea comes to mind, roughly define it, and roughly estimate it. For example: As a user, I want to manage stock of all the items I am selling through Company’s POS. This can be discussed within various product teams, e.g. Domain Experts, UX, Architecture, etc., that provide additional input into it. Together, in a planning session, these teams could provide a rough estimate, but I would avoid it if possible. The team could, however, roughly time it — e.g. late Q1 or early Q2. This then provides other departments with a better idea of what is coming.
3. When such a big (in Agile, this could be called an Epic) item is planned, it needs to be broken down into smaller chunks — for example, into stories. I have a lot of problems in this area, and I will discuss some of them in the next chapter. But these stories can now be better estimated by the teams. Still, these estimates must be rough and treated as such. This is why points are used, not dates. The team knows approximately what its velocity is, and that implies how many stories it can complete in a sprint.
4. When teams set down to plan a sprint, they can better estimate each of the stories (smaller items). What the team commits to, for the sprint, needs to get done. The team is accountable here.

Interacting with Domain Experts

This is my biggest pain point so far. I have always had a hard time interacting with domain experts. I believe the reason for this is that, looking sort of metaphorically, developers and domain experts sit on completely opposite sides of the coin. One side is completely a-technical, and the other side is almost 100% technical. Concepts such as Domain Driven Design seem to alleviate things like this, usually by introducing things like the ubiquitous language.

Here’s where it gets fun.

Some teams introduce the concept of an Analyst. These analysts need to have a good understanding of the problem domain, as well as the understanding of how software works. Some, though, seem to have an invalid perception of what this role actually means. The output of such a role, should be a good description of the business processes governing a feature. This is derived from talking to (and understanding) the domain experts.

Lesson #5: Not everybody thinks the same way.

In my opinion, the worst case scenario is the analyst who becomes an architect. This means that instead of defining processes, he starts defining models and user interfaces. This is then given to the developers who are supposed to “translate” the Word document into code. This reduces the input developers are able to give to the features and therefore reduces their contribution. I am not saying developers turn in bad code, far from it! However, I would argue that developers are disconnected from the product, therefore they don’t really care about what they’re doing (e.g. if it will really help the user). So keep everyone focused on their core job, but also…

Lesson #6: Everybody’s opinion needs to matter, but (informed) decisions have to be made.

This is probably one of the most important lessons I have learnt so far. Agile teams have a flat hierarchy. And while everybody is aware of their own responsibilities, and often, everybody is an expert in their own field, people will generally have opinions about stuff that matters to them. So if the developers feel connected to the project (and a part of it) they might have suggestions on UX, on domain, on many other things. Listen to them. They want what’s best for the project!