Why do projects fail?

When leaders play emperor, projects go commando

“The townspeople, who had also been swept up in the deception, enthusiastically cheered for the emperor’s new clothes, despite seeing nothing at all. It is only when an innocent child spoke the truth, exclaiming that the emperor is naked, that everyone began to realise the extent of their collective delusion. Slowly, the truth spread throughout the city, and the emperor’s foolishness became apparent to all”.

— Hans Christian Andersen, ‘The Emperor’s New Clothes’

Not all problems were created equal. In many cases, there’s an obvious fix. In others, a solution exists but without a clear plan, progress is unlikely. Many problems faced by local government fit into neither of these categories. They are messy, complex, wicked. The problem is evolving and there is no obvious solution. Solving it may never be possible. Progress, or even maintenance is sometimes the best we can hope for.

Different types of problems require different approaches to project management. In 9 out of 10 cases, Basis projects use an agile approach. We make this choice not because we decry other project and programme methodologies (PPM). We work on messy problems, and for our purposes, an agile approach and its associated methodologies (like Scrum and Impact Sprints) are an effective means to an end.

Choosing the right approach to dealing with the problem is just one variable that dictates the likelihood of a project’s success. Are people who feel the pain of things not working well involved in the work? Does the change team have the capacity to work on the project? Do all stakeholders understand the intended outcome? If you accept McKinsey’s data on the subject, the long list runs into the hundreds.

Few would disagree that the presence of a leader who is invested in the project would sit close to the top of the pile. Someone who helps everyone understand the destination and creates the space for the team to wrestle with the challenge without providing the answers. Someone who makes themselves available to provide feedback and who sees it as their role to manage difficult interactions with peers when they inevitably crop up. Someone who takes time to reflect on their contribution to the success of the work, who regularly asks themselves: “What’s my contribution to the mess I seek to address?” (hat tip to Rick Torseth). In the case of agile projects, without this kind of leader, the team is destined to spin their wheels.

Tom Alexander is currently the Head of Change for People’s Services at Haringey. He and I have worked with one another in several guises and multiple projects over the last 6 years. Wherever we bump into one another, I can guarantee he’s doing messy work. When people get stuck and need someone to ‘be accountable’ or ‘to grip’ the work (whatever those terms mean), Tom is usually the first person people think of. We often have discussions about the conditions needed for the messy projects we are both involved in to succeed. Our conversations are animated and unpublishable. A recent discussion led to some useful insights about the leadership qualities required to deal with local government’s messy challenges and Tom agreed to write up some of his thoughts for this Blog. Here’s Tom going full Tom.

Here’s Tom

As a student, I worked on a building site for some time. It was a brutal induction into outcome-focused thinking. I remember the guv’nor standing over me bellowing at me about how to mix cement correctly and then pointing at a hole in an internal wall and telling me to “brick it up”. The rest was up to me.

I did a good job but it was a bit messy. However, he knew it didn’t matter what it looked like. Assuming I could lay bricks in reasonably straight lines, it would never be seen behind the layers of bonding, plaster and paintwork. He made sure I understood what was required, gave me enough information to get started and then disappeared to do something else. He trusted me with the job. I appreciated that. As far as I know, the wall continues to stand today.

This same man, on another job, taught me to paint a room professionally. He taught me how to ‘cut in’ without masking tape and spared no details teaching me proper technique. He told me it would be worth spending time learning the skill because “one day you will have to decorate your place. This will save you loads of money”.

He took ages demonstrating how to fill holes properly. He showed me what brushes to use for which paint or covering and in which order I should approach the job so that errors got covered up as I went. He explained the importance of washing my kit so that I could use it for years to come. It was a detailed, planned and methodological approach that added enormous value to me. Until having a child, I did not pay anyone to decorate which saved me a fortune.

Two problems. Different approaches were needed. Each was successful. The constant was having a boss who understood how to adapt his style depending on the challenge to be overcome. He was like that naturally and with everyone in his firm. Although I didn’t know it at the time, this was my first experience of an adaptive leader.

In Local Government, the primary differentiating factor between projects that are fulfilling to work on and deliver genuinely useful benefits and the rest, is rarely the choice of techniques or tools used. It’s not even the people working on the project. Both are important. Appropriate methods and a skilled and motivated team increase the likelihood of success. But the big variable is the leader.

When I see a new programme or project coming together I ask myself whether the Senior Responsible Owner (SRO) is up to the job. Are they able to adapt to changing circumstances and support a team through the ambiguity this creates? Or will they tell people to use concrete to paint the bedroom wall regardless of the problem they are trying to solve? I ask these same questions of myself when I’m the SRO.

Adaptive Leadership in PPM

I read an article published in the Harvard Business Review in late 2020 which resonated with me. It talked about the 4 A’s of Adaptive Leadership:

• Anticipation of likely future needs, trends and options.
• Articulation of these needs to build collective understanding and support for action.
• Adaptation so that there is continuous learning and the adjustment of responses as necessary.
• Accountability, including maximum transparency in decision-making processes and openness to challenges and feedback.

I’ve worked in and led Programme Management Offices and have been responsible for large projects and programmes. I’ve seen many of the programmes succeed and others fail to deliver the benefits they set out to achieve.

Rather than fretting about methodologies or developing processes and plans in minute detail, I believe that local authorities would get 10x the bang for their buck investing in getting SROs comfortable with the 4 ‘A’s above.

Having an SRO that is forward-looking, sets clear problem statements, accepts that things change as we learn and genuinely takes and gives accountability (not just responsibility) is how the magic happens. There is more in the HBR article about how to do that in a project but you can read that for yourself.

Adaptive Leadership

Ronald Heifetz is a leadership guru and academic who has written extensively on teaching leadership as well as advising Governments and other organisations on how to build leadership capacity and capability. His view is that:

“When change requires you to challenge people’s familiar reality, it can be difficult, dangerous work. Whatever the context, whether in the private or the public sector, many will feel threatened as you push through major changes. But as a leader, you need to find a way to make it work”.

This strikes a cord as it is such a simple statement that reveals significant complexity. Here’s why:

• In Local Government, the problems we now face always challenge “people’s familiar reality”. We have been managing increased and more complex demand for over a decade whilst seeing funding continuously reduced — and the goodwill afforded to much of the public sector during the pandemic largely passed Local Government by.
• Against that background, it can feel personally dangerous and threatening if you are either accountable or responsible for a programme or project. You are going to carry the organisational expectations of those above, below and alongside in a specific and concentrated way, and that’s before we even talk about what residents want to see.
• The whole point of a project is to “push through major changes” and only the very best leaders are comfortable with terminating or resetting a project if it looks like it is not going to deliver appropriate benefits. It is one of the biggest problems in the public sector that people invest so much time and emotion setting up programmes and projects that they then go to great lengths to avoid closing or killing them off if the time comes to do so.

For us folks who are paid the big bucks to be accountable for delivering major change programmes or projects, what must we do and how must we behave to increase the chances of success? I think a good start is to adopt the adaptive leadership behaviours advocated by Heifeiz and outlined by Liz Lockhart Lance below:

1. Get on the Balcony
2. Identify the Adaptive Challenge
3. Regulate Distress
4. Maintain Disciplined Attention
5. Give the Work Back to the People
6. Protect Leadership Voices from Below

Most of us either want to go into our shell or micro-manage when the heat is on. But the whole point of deploying a project approach is to leverage the skills of a range of people to make progress on a challenge that an individual can’t solve. For programmes and projects to both deliver the right impact in the right way, the teams (who are not necessarily bound by traditional line management obligations) need leaders to demonstrate the above by:

Being visible and painting a compelling picture — but in a way that acknowledges the organisational culture and conditions people are working in.

If I, as a leader, have said things like “this is a real priority for me and I need you to deliver but I know that department X or organisation Y can be a bit of a pain so you need to find a way to manage that”, I am a delivery risk.

Help your team set an evidence-based problem statement, help them understand what the Minimum Viable Product needs to be, ensure they know they have permission to exercise their judgement and create an environment where people feel psychologically safe enough to call out issues and poor practice that threatens delivery.

If I, as a leader, have said things like “I need a Project Brief, a Project Plan, a RAID Log, a PID, a RACI, a MoSCoW and a Benefits Map” but the project team still cannot articulate the point of the project in a sentence or two, I am a delivery risk.

As Heifetz suggested, this stuff is personal. People want to see change happen but are less enthusiastic when having to change themselves or something they have built. Some of these people will be your seniors, direct reports or peers. What they supportively said in your most recent management meeting may not represent how they feel right? They need help with their loss (whether perceived or real) and it requires a personal touch.

If I, as a leader, have said things like “Yes I know person X hasn’t come to the last 10 project meetings, does not give apologies and won’t send a substitute but they are very busy so make sure you send them the papers and offer to meet them separately”, I am a delivery risk.

Do not sugarcoat the scale of any challenge with the team. They will be acutely aware of how hard the project journey is and will feel more exposed than you do. You need to make sure they understand you are under pressure (perhaps for different reasons) but that if they take the challenge on, you will give them air cover to take managed risks and try new things.

If I, as a leader, have said things like “my door is always open” or “we are in this together” and then don’t make the time to speak with the project team until the next Board meeting, I am a delivery risk.

You have recruited a good team so they don’t need you to hold their hands or to do their work for them. They need you to set the parameters and then check in with them in a way that is meaningful and appropriately challenging. Being a programme or project manager when your sponsor or SRO never has a chat over a coffee with you is a lonely place to be, especially where you have no direct line management connection.

If I, as a leader, have said things like “I don’t feel like I have enough grip on this programme/project, can you send me a briefing/highlight report/gateway report about it”, I am a delivery risk.

Recognise the privilege of leadership. Yes, you are a talented person but there are people within your shadow that are just as intelligent (or more so) and/or just as committed (or more so) and just as creative (or more so) than you. They will have some very good ideas and if you are not hearing any, the project environment is broken. Your role is to incubate leadership voices that are not paid the larger salaries or hold the more prestigious job titles. If you don’t, you will miss something important.

If I, as a leader, have said things like “Yes we do need to do some co-production but the Board meetings get into very detailed discussions and person X or group Y won’t understand what we are talking about so let’s come back to them later”, I am a delivery risk.

And, to really overdo the analogy, if you have ever said these kinds of things to programme and project management colleagues around you — and they haven’t called you out for it — then you rule a city where your townsfolk are too anxious to tell you that you are wandering about naked.

If you’d like to conduct a quick stock check of how adaptive your current approach to leading projects is, this scorecard will help.

If you’re anything like me, you’ll need a spoonful of sugar to make the medicine go down.

In reading Tom’s thoughts I recognise my own failings as a leader on several projects in the past. Some of his reflections made for uncomfortable reading. But leading change in public services is an uncomfortable business.

I often ask folks we work with on projects what good leadership would look like for them in the context of our work together. People often say things like good grip, accountability, and ownership of the work. I understand these words, but they mean different things to different people.

Not all of Tom’s reflections will sit comfortably with everyone. But I know that when I work with leaders who:

• Create the space for teams to wrestle with the challenge at hand
• Ensure people know it’s OK not to have all the answers
• Expect friction and see it as their role to work through it
• Make it possible for many voices to be heard and to adapt to new learning when it emerges

Our projects deliver better outcomes and are more fulfilling to work on.