How I was forced to do Dark Scrum

Some people may not have experienced imposed Agile. I have: it was awful and I hope it never happens to you. This is my story of how I ended up forced to do “Dark Scrum”.

The context

Many years back I was a developer in a Scrum team which was tasked with integrating with a new internal API. My team would work with an existing non-Agile team for a new team size of around 15 people. We had a new Agile coach who directed how we would work.

We had 2 goals:

1. Show this team (and their manager) a better way of working by being Agile.
2. Deliver the project scope on time and on budget.

Estimating the unknown

Our project manager wanted to know how long it would take to complete the project, so our Agile coach directed us to estimate user story sizes on the existing backlog. We said we couldn’t estimate the story sizes until we had started to work with the new API but he just wanted a “rough” number to “help with planning” and that he wouldn’t hold us to the estimations.

Once we had the total size of the backlog in story points (never mind that at this point we hadn’t coded anything, so had no sense of how much time a story point was a proxy for), our project manager then contracted with the other team’s business exec based on story points. Each story point had an economic value (of $100) and the success or failure of each sprint was tied to how many story points we had delivered. In other words, we had a target of a certain number of story points per sprint, which we had to meet or work overtime.

Story Point hell

This is an unhealthy way of thinking about story points and overtime, which I pushed back on, but was overruled (“If the team is bad at estimating then they need to learn how to get better, and working overtime will help that” was the rationale — never mind what was said earlier about not holding us to the estimations). Story points are meant to be an internal-to-the-team measure of the complexity of a user story in order to have a useful baseline of talking about a story’s complexity and getting a sense of what a realistic amount of work is which could be accomplished in a sprint. So when after our first sprint planning we decided what we felt we could accomplish our Agile coach said it wasn’t enough and asked the team to commit to more. I pushed back on that, hard, as it felt like manipulation (which it was). The coach disagreed, saying that this was how he liked to motivate people — that it would “stretch them to do more”.

This put massive (and unnecessary) pressure on sprint planning and each sprint, because if we didn’t deliver the “committed to” story points it put the entire contract at risk, so the team had to work overtime. The result was predictable: since story point sizing was no longer about a useful way of surfacing complexity and figuring out how much work we could realistically get done within a sprint, sprint planning became a high stakes stressful game, taking longer and longer and resulted in open (and unnecessary) conflict between team members.

Our Agile coach didn’t mind the conflict. He said that we were forced to work this way because of the contracting and there was nothing we (or he) could do about it. We just had to suck it up and work overtime if our estimations weren’t accurate.

The project manager doesn’t get it

Deeply concerned, I met with our manager and Agile coach, reminding them that we had made up the story points based on complete guesswork. I also showed that the team would start artificially inflating story points due to Goodhart’s Law:

“When a measure becomes a target, it ceases to be a good measure”.

They responded by saying that it would be unprofessional for a developer to change their estimate after the fact, and that we could still trust the estimates, and that we would continue working in the same way.

More Dark Scrum

This pattern was to be repeated: I raised concerns about our way of working to my management team, and they ignored me. Some other examples:

• We couldn’t choose our own team size or structure teams differently (it had to be one team, even if standups were taking 1/2 an hour).
• Standups were a status report to the Agile coach more then anything else, and we weren’t given leeway to run standups how we wanted to.
• I coached the team on WIP limits and when the team asked to adopt them, the Agile coach said no.
• He wanted to stop retrospectives because “we don’t need to put effort into improving, we should just keep working.”

Each time I raised concerns to the Agile coach and the project manager, the answer was always the same:

“We have contracted for one Scrum team and delivery based on story points; we promised that we’d ‘Be Agile’ and so we cannot change how we work.”

Double binds

I began to feel more and more stressed. I now know that feeling of craziness is the symptom of being put in a double bind: an

“emotionally distressing dilemma in communication in which an individual (or group) receives two or more conflicting messages, with one negating the other.”

The team was put in various double binds by our project manager and Agile coach:

• We know you can’t estimate the complexity of integrating with a new API; you have to estimate the complexity of integrating with the new API.
• We won’t hold you to your story point estimates (so please just give us a gut-feel estimation); we will hold you to your story point estimates (so you’d better be 100% accurate with your estimations).
• We are working in an Agile way and are therefore committed to improving; you can’t work how you want to because you’re not allowed to improve.
• You should only commit to a realistic number of story points for a sprint; you have to commit to a certain number of story points.
• We want to work at a sustainable pace; if you don’t meet your story point targets you will work overtime.

Our Agile coach was in a double bind:

• Coach the team to be better (which implies change); the team cannot adapt the way they work because it will show that we weren’t doing it right from the start.

The confrontation

At this point I decided that the Agile coach was clueless about Agile (I’d given him far too much benefit of the doubt as he was a much older man with vast experience leading software teams). I wrote down 20 specific things where he’d got the team to do the opposite of what I’d consider healthy Agile practice to be, and I met with him and we went through the list one by one (credit to him for having this meeting in the first place). He agreed with me on every point (!) and said that actually he wanted to coach the team to do the healthy thing, but that the contract which was in place prevented him from doing so.

He also said that one of the promises the project manager had made was that our team would show the “right” way of working, and that we had to get that right upfront otherwise it would look like we didn’t know what we were doing. No inspect-and-adapt for us as “improvement” was an admission that we didn’t know what we were doing when we started.

In other words, our new “Agile” way of working had to be in a Big Upfront Design way — that “Agile” was about a predefined, unchanging set of specific patterns that would “work” as long as we followed them.

The result

I burned out and resigned (even now, years later, I feel stressed thinking about it). The project didn’t go all that well either.

Where did it go wrong?

It went wrong on minute six of day one. Here’s how.

Scrum is meant to quickly surface any pain in the system because of its transparency. It will actually appear to make things worse for a while but all it’s doing is drawing attention to the symptoms of a deeper problem so that it can be dealt with. “Working in an Agile way” is not just about how the development teams choose to structure their own work; it’s about getting all stakeholders to address the (initially unknown) issues which would stop a project from being delivered. It’s for this reason that Scrum highly values transparency.

The first symptom emerged within the first five minutes of day one when we said that we couldn’t estimate something we have no knowledge of. Our PM wanted to see how long the scope would take to build in order to manage our risk, so it made sense to estimate the backlog in terms of complexity (and of course we then should have run a few sprints to see what the actual throughput of the team would be and then do something like run Monte Carlo simulations for continuous forecasting so we could see how on track we were each week, but that’s a different point).

Instead of forcing estimations to happen anyway, a skilled Agile coach would have used something like the Cynefin model to realise that we were in the complicated domain and so needed expert assistance. In fact, since no-one had integrated with this API before we were actually in the complex domain — the domain of unknown unknowns. The order of action there is probe — sense — respond, so using a safe-to-fail probe (see Liz Keogh’s excellent talk) we could have tried a small integration with the API. Only once we had done that would we have more information about the complexity of the work.

In the meantime, the Agile coach would have discussed this with our project manager who would then have to have a difficult conversation with the other team’s business exec:

“Given that no-one has done this before we can’t tell you how long it will take. I know that your current team hasn’t delivered and you want certainty but it’s too soon for that to emerge. Our plan is to run a small experiment to see what we learn. Once we’ve done that we’ll have more information about the domain. We’ll report back within 2 weeks.”

Her challenge would have been to shift the expectation away from simplistic “This is how long it will take” thinking to “This is a complex problem and we need a different way to approach it. It will require a high level of trust. Are you happy to start this process with us?”

All of this could have emerged from minute six of day one if the Agile coach had embraced the first (and most important) Agile value of “individuals and interactions” and listened to the discomfort of our team members. But instead “Agile” was imposed on us which lead to Dark Scrum and disastrous consequences.