“Push Right” — From Agile to Antifragile

Something’s not quite right with IT project management today. It is better than it was but it is still not quite right.

We have methodologies and training courses aplenty and yet the profession is still dogged by project failure.

Indeed in such circumstances, one can’t help wondering about those methodologies and training. In implementation, they are often too mechanistic and they downgrade the key human faculties of intuition and common sense by trying to reduce everything to a process. This pleases those in higher management who claim to want (but don’t always actually want) structure, consistency and measurability.

But life doesn’t work like that and nor really does project management.

I used to think that knowledge of a well-established project methodology was necessary but not sufficient to be a good project manager. At some point that turned into “necessary but nowhere near sufficient”. But what if it’s actually worse than that? What if a project methodology is necessary but paradoxically sowing the seeds of project failure?

Think of Prince2: a well-established methodology born from that well-known bastion of successful projects the UK government, and now demanded of all PM practitioners in the UK skills market. I’ve seen situations first hand where Prince2 Project Managers are too busy churning out or updating project documents and focusing on the issue du jour to realise (or to baldly state) that their project is essentially undeliverable. And these are not novice PMs.

Now, one might claim that it wasn’t Prince2 (or its offspring MSP and P3O) nor the PMI that introduced this risk but an over-zealous interpretation by practitioners — the project managers. But that’s ducking the issue. These methodologies have a life of their own and indeed an entire industry of their own. The raison d’etre of the groups backing these methodologies is to see them implemented. It would be surprising if project managers steeped in such methods didn’t have an impulse to fully implement them.

So no, diverting the blame from the methodologies to the project managers won’t do. Methodologies have their place - they should be learned and internalised by practitioners. But thereafter project management is an art that needs to be almost relearned, applied and finessed in the real world, and that often means a very flexible interpretation of “the manual”.

And then there is risk management.

Does anyone really know what the risks are to their project? Sure, we can spot the obvious ones, write them down in a neatly formatted table and then dream up some probabilities and impacts alongside a sentence or two describing what we might do if they became are reality. We may even construct more elaborate tables attempting to measure inherent and residual risk, proximity, and other column headings — the more the better, apparently.

But the process is often pointless. In the course of managing many real and successful projects and programmes, my risk log often only exists to show to anyone who asks “Can I see your risk log?” In other words, it’s a box-ticking exercise. It can also make risk management resemble a science and in so doing strip it of professional judgement, meaning and value. And I’m not all that convinced senior managers take much notice of it anyway.

In the technology world, we are often dealing with complicated systems that no one person fully understands. And the more complex the system, the greater the law of unintended consequences. In such circumstances, how can project managers and their teams possibly evaluate the risks of replacing or upgrading such systems or introducing completely new ones?

That’s not to say risk shouldn’t be considered. There will be some obvious bear-traps that need highlighting. But for the project manager to go beyond that and guess what may or may not happen is often making a rod for one’s own back: it can send a signal to senior management that suggests all risks are known, thoroughly considered and actively managed, which is seldom the case.

In my experience, a full examination of assumptions is a more fruitful exercise but senior managers are even less bothered by that, and the project methodologies (and some project managers) have been slow to focus on assumptions.[1]

And so the default position of too many project managers, supported by a project methodology and egged on by over-active PMOs justifying their own existence, has been to “apply the manual” to projects. The core PM principles of common sense, communication and intuition born of experience are either smothered, or in the case of communication perverted to an extent that all effort goes into the wrong sort of communication — usually management reports and endless slide decks. There are too many examples of project management process wraps being thrown around a cloud of dust — a set of required products that can’t possibly be delivered in the way envisaged. Noting this “risk” in a table on page 17 of the PID doesn’t cut it.

The other point about risk management within the project management context is that it is often a forlorn attempt to restrain change, thus nurturing a PMO culture of risk aversion. In today’s world where enterprises themselves have to become more nimble and take risks rather than avoid them, having a project management function faced in the opposite direction is not exactly going to improve its influence, nor will it be seen as the ‘go to’ function for all the changes an organisation may want to (or may have to) bring in.

But here’s a thought: one of the biggest commercial failures in history happened in 2008, not in a devil-may-care industry with a startup culture but in one of the most regulated and risk-aware industries on earth: banking and financial services.

Regulation, process, and risk management are clearly not the “obvious” answers that many assume.

The position of Agile

The Agile Manifesto of 2001 was essentially a disruptive technology for the project management industry and has been gradually spreading into it for nearly fifteen years. It has been a hugely welcome change. Indeed we have reached the stage where happily there is such a thing as “Prince2 Agile”.

Furthermore, 2013 saw the launch of the PMO Manifesto in the UK which should have set a few alarm bells ringing among traditional PMO practitioners. The core principles of the PMO Manifesto being:

* Individuals and Collaboration over processes and tools.

* Enabling Change over restraining Delivery.

* Forecasting the Future over reporting the past.

* Striving for Improvement over accepting the status quo.

Note the relegation of processes, of reporting past events and of restraining delivery. One can also see deliberate parallels with the original Agile Manifesto’s core values:

* Individuals and interactions over processes and tools

* Working software over comprehensive documentation

* Customer collaboration over contract negotiation

* Responding to change over following a plan

Note again how processes and documentation are relegated in priority, including even the following of a plan.

Agile, however, went as far as giving us some new working methodologies like Scrum and DSDM Atern along with a set of now widely-accepted techniques, for example:

a) The time-boxing of work into smaller manageable parts

b) The frequent stand-up meetings

c) Regular delivery

d) Highly collaborative teams with flatter structures, including (particularly) stakeholders

e) Constant team learning through frequent retrospectives

f) Fixing the timescale but flexing the requirements. Well, in theory anyway…

The effect of this approach was to give projects momentum; deliver early and iteratively thereby containing risk including outright failures; and enable rapid team formation, communication and high performance.

Agile was born from the software development world and was/is well suited to such projects. Attempts have been made to extend its techniques into IT infrastructure projects with their third party suppliers and longer lead times (and traditionally risk-averse operations teams). There are however practical limits on how far Agile can go into this space but it has spawned other Agile offshoots, most notably the DevOps movement that brings together the traditional silos of Development and Operations to deliver faster and better. And as we have seen, Agile has now percolated into Prince2.

But while Agile’s reach is now wide, we can and should push the boundaries further still.

Agile projects are structured to tiptoe through a landscape of risk one short step at a time and thus avoid anything calamitous like a complete failure to deliver what the customer wanted. To go further we need to change our attitude to the risk landscape: to move from simply avoiding or containing the risks/issues that we can’t really predict to accepting and welcoming them such that we learn/grow not just the project team but the entire enterprise.

Secondly, we need to work to make the risk landscape less dangerous to delivery, thereby allowing projects to proceed with greater pace and confidence. And we need this to be an ongoing process.

The Antifragile Concept

I read Nassim Nicholas Taleb’s “Antifragile” book a couple of years ago but it is very relevant to this debate.

Taleb defines fragility as things that are negatively impacted by shocks. Most of us would define the opposite of fragility as robustness — things that withstand shocks, but Taleb notes that the opposite of fragility is actually where things are positively impacted by shocks. And there is no term for this so he invents one: “Antifragility”.

He supports his argument about the power of antifragility with many examples of where stressors improve a system, especially living or human-devised systems, and how the absence of stressors do the opposite: cause atrophy. With fragile systems, stressors simply cause breaks that are very difficult to recover from. He therefore argues that we should aim for antifragile systems that don’t simply break and cause anguished post-mortems, but positively flourish as a result of the experience.

As an ex-hedge fund manager and observer of the financial world, Taleb is scathing of supposedly advanced financial, economic and political cultures but his criticisms inadvertently find an echo in the project management world. For example:

“A complex system, contrary to what people believe, does not require complicated systems and regulations and intricate policies. The simpler, the better. Complications lead to multiplicative chains of unanticipated effects. Because of opacity, an intervention leads to unforeseen consequences, followed by apologies about the “unforeseen” aspect of the consequences, then to another intervention to correct the secondary effects, leading to an explosive series of branching “unforeseen” responses, each one worse than the preceding one.

Yet simplicity has been difficult to implement in modern life because it is against the spirit of a certain brand of people who seek sophistication so they can justify their profession.

Risk management as practiced is the study of an event taking place in the future, and only some economists and other lunatics can claim — against experience — to “measure” the future incidence of these rare events, with suckers listening to them — against experience and the track record of such claims.”

Later in the book, Taleb briefly covers projects and has a dig at consulting firms and modern business schools that “teach something called project management” and notes how IT projects have a knack of over-running. He further notes how major projects of the past were much less likely to over-run, commenting that…

“Black Swan effects are necessarily increasing, as a result of complexity, interdependence between parts, globalization, and the beastly thing called “efficiency” that makes people now sail too close to the wind. Add to that consultants and business schools. One problem somewhere can halt the entire project — so the projects tend to get as weak as the weakest link in their chain (an acute negative convexity effect). The world is getting less and less predictable, and we rely more and more on technologies that have errors and interactions that are harder to estimate, let alone predict.”

Taleb’s book is thus a call to arms: to push back on what he sees as a fragile, self-defeating industry of management layers, “experts”, and process, along with a simultaneous call to recognise the unpredictable world in which we live and to work in a more antifragile fashion.

He wants us to push to the right of his fragility spectrum: Fragile — Robust — Antifragile.

And project management methods fall along that spectrum, with traditional waterfall projects (and old Prince2) sitting at the fragile end of the spectrum and Agile positioned somewhere between Robust and Antifragile. That gives us an opportunity to ‘push right’ and advance further towards antifragility.

Why do so?

Because wouldn’t it be better to stop creeping through a risk landscape that we can never really understand or predict and instead go forward with greater confidence knowing that we have a project organisation and enterprise that will flex and grow when risks/issues hit us? Wouldn’t that make us more accepting and aware of risk than an environment where we relegate risks to a neatly-formatted piece of fiction that will gather dust until it is waved back at us when an unpredicted disaster strikes?

Furthermore — and this is really important — the creation of an organisation that operates according to Antifragile principles will have the effect of dispersing risk such that it can be managed far more effectively when it arises. Words like devolution, dispersal and decentralisation are very much a part of the Antifragile mindset.

Now briefly consider a traditional IT project management scenario that should be familiar to many: A situation of a “good process-oriented project manager” managing “from the front” such that all project-related matters come through him/her and no team member does much without involvement and direction from the project manager. This is what I call a classic “star structure” in a project (perhaps more accurately a “hub-and-spoke structure”) whereby the PM is the centre of everything.

The problem with this is….the PM is the centre of everything. They take on too much; they feel the need to second-guess everyone on the team — in extremis they may even take on the tasks of other team members; no conversation can take place without them directing it; they get lost in the detail; AND they update a constant stream of “methodology” documentation.

Such people soon demonstrate that they themselves are the cause of project fragility. But there’s no doubt they are doing their best to “follow the manual” and may even be held up as a methodology champion within their organisations, which is why my instinct tells me project methodologies and their followers are paradoxically sowing the seeds of project failure.

Antifragile project management

How then do we create an Antifragile project management culture?

I have already touched in the dispersed nature of antifragile systems. A fuller list would include:

1. Ability to flexibly respond

2. Ability to fail fast and recover

3. Decentralised/flat structures

4. The constant need to simplify (complexity can be the enemy of antifragile systems)

5. Dispersed risks

6. Embracing risks as opportunities for learning

7. Support for tinkering/trial and error

8. Continuous and fast learning with effective feedback loops

Some of this can be found within an Agile project but as was touched upon earlier, one soon hits limits when trying to apply Agile to non-software development projects. Agile project management must consequently be seen as a subset of a broader Antifragile approach with the latter having wider application. We need other approaches (and tools) to help us pull other projects under the Antifragile umbrella. And some of these approaches are actually very old, one might even say ‘timeless’, but still we forget them.

My own “culture checklist” for an antifragile project is therefore:

1. “Everybody leadership”. An organisation chart of an antifragile project would show a network of team members and stakeholders, each connecting to everyone else. The project manager role may not even explicitly feature on it. That role would be the whitespace upon which the structure is drawn — keeping the communication flowing, keeping the whole thing glued together and maintaining momentum. Lao Tzu’s ancient quote about the best leaders being those where the people/team think they delivered it themselves is very pertinent here. For the project manager, so is the concept of “servant leadership”. Agile goes a fair way to achieving this because the Agile project manager has a much more facilitative role. This model of “everybody leadership” can however be transferred to any IT project.
2. Simplify Simplify Simplify. There are enough complications in a technology project as it is and the PM should be the champion of simplification, which is one good reason why it is not such a great idea to use “Technical PMs” well versed in every technical detail of what they are delivering. That may seem counter-intuitive but no one ever made progress by doing the same thing and expecting different results. Instead what’s needed is an inquisitive non-technical PM who can make sense of the project’s complexity with less prior knowledge and therefore fewer inherited assumptions.[2] The first thing one tends to do on the toughest projects, including those that need rescuing, is to take them back to the basics: what are we actually trying to achieve here, what really needs delivering and what are the blockers? If we tend to simplify and strip out the noise when a project is facing a crisis, why not all the time?
3. Communicate Communicate Communicate. As a PM, one must be in the thick of the project at all times in order to live and breathe it from hour to hour, minute to minute….which is why one should try to avoid working from home.[3] Communication is the most important facet of a project as one of the best books on project management — Richard Newton’s “The Project Manager” — stresses. The PM must ensure that communication is constantly flowing and not with his/her constant approval — we want to actively avoid that old star structure. A whole book could be written on project communication but ultimately what we want is to promote and facilitate project communication without churning out numerous reports, slide decks and endless invitations to hour-long meetings.
4. Be light on your feet. Do not get bogged down in process or documentation or endless planning and re-planning. “Travel light” and do what is absolutely necessary (which is not always the same as what is wanted by PMOs). As well as tuning the project management role itself, this will require education of senior business managers some of whom demand a baselined GANTT chart almost as soon as the project starts. The idea of being light on your feet ties into the above two points about simplification and “everybody leadership”. As a project manager, you can be much more nimble if the right people are making the decisions, the communication is flowing unhindered among team members, and while you are constantly tuned in to all of this, you are primarily focused on keeping the team and project focused on its goals.
5. Continuous learning. How often have lessons really been learned by the PMO or the enterprise such that things then change? Not often. How often is the team and enterprise actively learning, week by week, as the project progresses (outside a properly functioning Agile team)? Not often either. We need proper feedback loops that are rapid, regular and genuinely foster change in the project and the enterprise. Think of the aviation industry. If something goes wrong, the lessons are fully investigated, communicated and acted upon across the whole industry. Too often in IT projects, the lessons learned activity is either a box-ticking exercise at the end of the project when everyone has lost interest, or it is a diligent exercise capturing and cataloguing lessons learned, filing them in a PMO area….and then forgetting about them. An Antifragile project cannot absorb stressors and recover/grow from them if it isn’t actively and ruthlessly learning on the job.
6. Continuous testing/risk seeking. This is straying into the realm of Antifragile tools rather than Antifragile culture but continuous testing/risk seeking both instils and reinforces an Antifragile culture of growth through stress. A good example is Netflix’s use of “chaos engineering” or as they put it “injecting failure into production to ensure our systems are fault-tolerant”.[4] Netflix have an algorithm called “Chaos Monkey” that essentially kills production processes randomly to make sure the service can survive common failures without any impact to customers. It allows the systems and the Netflix team to learn and improve all the time.

These ideas (and tools) require further exploration but one can see from the checklist that Antifragile is not quite as constrained as Agile in its application beyond the software development world. It is a superset of characteristics above Agile and thereby more readily transferable. It is also a mindset — a set of cultures and attitudes that require the whole of the IT function and even the enterprise to be reengineered to support.

The Antifragile approach doesn’t remove the need for a traditional project methodology but it undoubtedly relegates it. The best parallel I can think of for the project methodology within this culture is the university degree: useful, eye-opening, strengthening and you never really forget it, but ultimately you move beyond it when the real world arrives.

Antifragile embraces the real world and its knack of not only recovering from shocks but growing from them. The project management profession should embrace it too.

[1] There have been valiant attempts to address this, not least by risk management expert Keith Baxter and his ABCD method — Assumption-Based Core Dynamics — which has assumption analysis at its core. However it is still far from widespread adoption and sadly nowhere near being recognised by senior business managers.

[2] We are sadly a million miles from this in the UK — the entire PM skills/labour market is predicated on “previous knowledge of Technology x, y or z” which is sometimes necessary, but often not.

[3] ‘Breathing the project’ is clearly more difficult when working on multiple smaller projects, making that a skill in itself. But smaller projects tend to be lower risk and thus the balance of PM focus versus what could go wrong is different.

[4] Netflix techblog on chaos engineering: http://techblog.netflix.com/2014/09/introducing-chaos-engineering.html