‘Sugarcoating doesn’t help’: Agile coach takes a strong stance on how to fix government IT
Dave Witkin leads a small Washington D.C. area consulting firm focused on helping government agencies build better software using agile at scale. Over the last 25 years, he has used agile approaches to help agencies avoid failed IT programs and become more productive.
Dave is a member of AGL and contributes to the AGL Live working group, which helps drive meaningful conversations with innovative government leaders. In this interview, Dave explains why he is direct with agencies and his fellow contractors about how to fix government IT.
Why are you part of AGL and what inspires you about this community?
I’m part of AGL for a completely selfish reason: I want to help the government avoid failure on software programs, and I think the AGL community can help me do that. When I tell people my focus is helping government programs avoid failure, some say I’m taking a ‘negatively focused’ view. They say I should focus on something more positive, like making the government better at delivering software.
And look, I’m all for the positive. But first, we have to stop the bleeding. By not talking about the negative — the massive failures — we’re sweeping the magnitude of the problem under the rug. For example, by some estimates, the government spends $30B each year on failed software projects. And by failed, I don’t mean software efforts that were late, over budget, or didn’t deliver what the customers expected. I mean cases where the government spent huge sums — sometimes hundreds of millions of dollars — on a program that never actually put any working software into use for customers or citizens.
That’s 30 billion dollars that can’t go to delivering services to veterans, improving education, protecting our active duty military, or anything else we would like to improve in our country. That hurts me, the taxpayer, and it hurts others. I keep talking about stopping the failures because over the past 25 years I’ve found that sugar-coating the issue doesn’t help anyone. People mean well when they soften the message, but it is counter-productive.
I keep talking about stopping the failures because over the past 25 years I’ve found that sugar-coating the issue doesn’t help anyone.
So that’s why I’m excited about AGL. There is a community of people here that are just as passionate as I am about building a better government by spreading awareness of the agile mindset that can help us start to fix things.
What’s an example of a project where you see potential for an agency to shift towards agile methods as a way of preventing failed programs?
I’m currently working on three federal government programs — two focused on delivering enterprise-wide software solutions, and one focused on transforming an agency with 100,000 people to use agile at scale. One of the software-focused programs is an excellent example of why I’m so persistent about getting to the root of the problem when a project is failing. This particular government IT program has already failed — not once, but twice — meaning tens of millions of dollars spent, a suspended program, and no working software. This is a situation where agile methods could help turn the tide, but it’s been a huge struggle to get buy-in from the agency.
Why do you think some agencies continue to use waterfall when the projects keep failing with that approach?
That’s an excellent question. I think human behavior can be incredibly ‘sticky’ and because we are bombarded by information nowadays, it is easy to tune out data that doesn’t fit with our existing beliefs. It takes a comprehensive campaign to change people’s views — including an overwhelming set of evidence, emotional appeals, and a clear path to enable change.
Human behavior can be incredibly ‘sticky’ … it is easy to tune out data that doesn’t fit with our existing beliefs.
On the government program I mentioned above, even though everyone on the project knows about the past failures and really wants to fix them, everyone has a different view about how to do that. For example, the lead system integration contractor has always done these projects in a typical waterfall fashion, so they tell the government that waterfall is the best practice. And the lead PMO contractor (another huge, well-respected government provider) agrees and says waterfall is the ‘safest way.’ They say that all we need to do is tweak the approach a bit. And these two contractors are considered leaders at what they do.
What those contractors don’t say is that about 35% of the large projects using those waterfall “best practices” are failures — or that somewhere between 70% — 95% are late, over budget, or deliver less than promised.
Unfortunately, despite having a few certifications, most of the people who work for these contractors don’t really understand waterfall failure rates or what agile is all about. And that makes some of the people advising the government a big part of the problem.
Could we use a “lessons learned” or retrospective approach to show why waterfall doesn’t work?
I’ve discovered that surprisingly, in some cases “lessons learned” can be counterproductive. Why? Here’s an example. There was a lot of documentation on lessons learned from the prior two program failures I mentioned. Do you know what was never mentioned as a key source of failure? Waterfall.
It’s rare for government reviews of projects to pinpoint the whole methodology as the problem.
And that’s critical. It’s rare for government reviews of projects to pinpoint the whole methodology as the problem. Instead, the focus is on tactical suggestions — you need to manage your requirements better, you need to improve governance processes, you need more documentation — things that aren’t really the core issues. These are problems cited by people literally entrenched in waterfall nightmares.
What advice would you give to an agency that wants to avoid these recurring issues and find a better way to deliver IT?
I won’t pretend there are super-easy answers to everything. But here are a few places to start.
First: If a waterfall approach to software delivery is proposed, stop. Do not move further — even if every proposal or contractor tells you this is the way to go. Bring in an independent agile expert (not one of the huge system integrators) who really lives and breathes agile. That’s all they do, and they are true experts in their field. They are not trying to sell you $300 million dollars in integration fees that benefit their firms. Have them recommend an incremental path forward. You don’t have to choose what they recommend, but at least consider their suggestions.
Second: If a so-called “hybrid” agile methodology is proposed, stop. Do not move further. Here’s what hybrid agile usually means: “Agile is a great buzzword. Everyone is talking about it. But we don’t know how to properly implement agile on this program or we think it will be really difficult or we don’t have the right staff for it, so we’ll use some agile terms and overlay them on a waterfall approach.” This tactic is often called “Scrum-Fall.” It has become so problematic that the Department of Defense (DoD) recently released a document entitled Detecting Agile BS where they cite Scrum-Fall and provide some guidelines on how to detect it.
If you or your contractor don’t understand the root causes that make agile effective, it is easy to be seduced by a hybrid model.
There’s overwhelming evidence that programs using proven agile methods can provide massive benefits — including reducing failure rates by over 50%. But hybrid or Scrum-Fall methodologies often omit the key features of agile approaches that have made them so effective, negating most of the proven benefits. That’s a key here. If you or your contractor don’t understand the root causes that make agile effective, it is easy to be seduced by a hybrid model.
These ideas call for more conversation — how can people get in touch with you?
Assuming you are not a large system integrator who wants to send me hate mail, I’m always happy to connect. The best way is via LinkedIn, or you can email me at dwitkin.agl “at” packagedagile.com.
Join Dave Witkin and others who are committed to helping government work better. Become a member of AGL: https://www.agilegovleaders.org/join/
