Untangling Wicked Digital Problems
TL:DR — You can watch me give a talk on this issue at NUX5 2016. You may also be interested in attending the talk I’ll be giving at Talk UX in Boston on the 18th October. Find out more on the Talk UX website.
‘Wicked problem’ is a phrase coined in 1973 by professors Horst W.J. Ritter and Melvin M. Webber. They were looking into issues around policy and the design of urban spaces. They used the phrase to describe the types of problems that plague humanity, problems that are so persistent, so difficult and so changeable that they are wicked in nature. These issues, such as poverty and crime, are always tough to solve. Their nature changes, the context is totally different in one example to the next and the best a designer (in its broadest sense e.g. policy, urban planning etc.) can hope for is to minimise or tame these issues, rather than fix them for good. It’s also worth noting that attempts to minimise do not take away from the fact that wicked problems are often fraught with inadvertent consequences.
Over the last few years of working at an increasingly strategic level with digital and business transformation challenges, I’ve come to see many of the challenges organisations are facing as wicked problems. Not because they are dealing with the classic list of problems that face humanity (although governments, charities and not-for-profits are certainly combining digital issues with human challenges), but because digital world has created its own unique set of wicked problems. Issues that plague teams no matter the size of the budget. Whether a small tactical app improvement or wholesale service overhaul, its become clear that many organisations are unwittingly hosting conditions to are hostile to success.
Although it can feel hopeless for those on the coal face, I’d like to suggest that bringing these issues to light is the first step towards making a significant difference to how you tackle them. There is no panacea for digital wicked problems, but once you know what you’re dealing with, it is possible to be more focused on minimising the damage, both in the short and long term.
What are Wicked Digital Problems?
I propose that there are four key wicked problems. Each a troublesome bugger when taken on individually, but as a quartet, these issues are almost always catastrophic. So lets get into each one.
Legacy Technology
This should sure be obvious, legacy technology from the late 90s (and often older) is plaguing pretty much any business that doesn’t consider itself a start-up. These large infrastructure systems from the late 90s and early 00s (often out of warranty, no longer supported by the vendors) are monolithic, often written in inflexible languages (like C) and are perennially difficult to untangle from the core functions of the business. Replacing them is a nightmare, and creating new services that have to talk to them is equally fraught. As my good friend Dave Hrycyszyn often says, “your future is held back by the limitations of your past”.
We cannot blame Microsoft, IBM and Oracle because their databases from 1993 are not compatible with the demands of one page apps, ubiquitous mobile devices and cloud technologies. They could not see the future (no matter how many times they sell themselves that way). Even more recent well known clangers such as Windows XP and Sharepoint 2007 have been exceptionally problematic in pretty much every organisation I have found them in.
What is less accepted as legacy tech are the things we’ve bought and built more recently. We so called modern digital folk don’t recognise that our own solutions, things we’ve built as part of our new progressive digital cultures, may themselves be problematic legacy tech for someone else.
One example of how this happens is when we take on projects that do not deal with the original legacy challenge (an old database for example) and find interesting hacks and workarounds to append new functionality. The non-ideal architecture will need to be carefully documented for future maintenance (often not done) and whilst the hack may suffice for the current release environment, who knows how much more hacking will be needed when there is a platform update on say iOS or Android?
Essentially, we are constantly making new legacy tech challenges whilst we attempt to somehow avoid the catastrophic, fear scenarios of switching old things off. Equally, for every new super fast Agile project, there are bound to be gaps and things missing that will cause someone else a headache further down the line.
Poor digital literacy in business leadership
This one may be controversial but bear with me as its not about apportioning blame. Many established businesses have leadership whose expertise is not grounded in digital. Despite this, today’s C-suites and leadership forums are plagued with the language of ‘Disrupt everything’, ‘Go Agile’, ‘Transform or Die’, ‘ Be digital first’ etc. etc.
However, the rhetoric being what is is, c-suites are struggling to be sufficiently informed enough to make good decisions. Decisions around budget, timelines, procurement, vendors and so forth, are made based on limited knowledge and following the procurement practices of old.
One client I worked with wanted wholesale digital transformation, but their core data asset (the main thing their business was built on) had been bartered away in a joint venture a couple of years earlier and was not available to them in the way it needed to be. This was probably a good financial decision at the time they made it. The problem is, if you do not control your data, you cannot control your transformation. And they didn’t know that.
Its important to consider how sensitive and perilous business leadership can be. You have to make decisions about many things you are not an expert in, and your only hope is to have the right inputs and advisors. However, many businesses have taken a wrong turn by buying into start-up culture. They listen to expensive consultants and Silicon Valley Experts over the practical, day-to-day experiences of their own staff (maybe talk to the guy fanning the server thats always on the verge of overheating).
IT Leadership are due some of this criticism, as many of them have become experts at buying Microsoft (and increasingly Google) solutions wholesale, or outsourcing their costs (because they still see themselves as a cost) to ever cheaper near or offshore suppliers. They have lost touch with the realities on the ground.
When timelines and budgets are unrealistic, when promises are unfounded, when no one can explain endless overruns and missed targets I find it often goes back to the leadership missing critical insight with which to make better decisions.
Finance practices
One of the most important things I always strive for as a practitioner and in the design teams I run, is a commercial conscience. Many will say this is because of my agency background. But even when I have worked in-house, I feel it is important to understand how the money works. Because this will ultimately be the variable that all decisions are based on. Strong financial practices are what successful businesses are based on. It’s why so many accountants and commercial folk end up as MDs and CEOs.
The issue is that the most popular financial practices of recent times, such as best value procurement and zero budgeting, are having a disproportionately negative affect on how businesses can execute digital projects. It isn’t the case that good financial practice is at odds with a successful digital culture, its that many organisations are looking at digital transformation, but keeping HR, legal and accounts the same as they’ve always been. You can’t half transform.
For example, an old client of mine was extremely proud of their procurement, boasted that not a penny was wasted. It meant there were a lot of overheads with getting any money out of the business. Forms, line manager sign-off and business cases. All good, traceable money management. Except when you’re doing a test project and need to buy 20GB extra space in the cloud. Something you or I can do in minutes, but they couldn’t do for almost 4-weeks. Four weeks when they had already hired a full design and build team, four weeks when we couldn’t not fully test the solution we were building. That four-week delay ultimately cost 6-weeks of re-work. Now you could argue that we should have waited to have all environments in place before starting right? But this is Agile!
It isn’t just procurement that is causing issues. Zero budgeting can mean senior and middle managers are not able to invest long-term, so they do a lot of small tactical projects, ultimately creating waste. And its not just in accessing budget where financial practices are having a negative impact.
Many businesses see IT as a cost. And when it comes to looking at value and return on investment, there is a struggle to reconcile against accounting methods that rely on analogue definitions of assets.
Lean and Agile practice
Now this may be contentious amongst my reading audience, but its important to be honest with ourselves about the limitations any process has. The Lean movement saved us all at a time of bloating project timelines and inefficient delivery (probably down to some of the issues I’ve already talked about above). Agile development gave us a way of pushing ahead fast, a solid set of ceremonies and behaviours that made it easier for us to avoid the bloat. By design, you just don’t have the time to do too much.
But somehow, although not intended, all the talk about start now, launch early, launch often and continuous improvement, has made people forget about what you might need to do BEFORE you start. As a former business analyst, I do not wish anyone the pain of putting together detailed requirements documents. But whilst the document itself was often a door stop, the activities that informed the content; exploration of the existing system, reviewing and agreeing new processes, these are critical when you’re not working in a greenfield environment.
I mentioned the client who had given away access to their data earlier, well when we started working with this client, we had managed to go through an entire discovery process (looking at user needs) and initial exploration (examining architectures) without realising just how significant the data issue would be. Ultimately, we ended up creatively working our way through (hacking) by building a set of cumbersome and inelegant workarounds that also delivered a far from ideal user experience. The only real problem we solved was saving face for the business leadership by launching a thing in only double the initial time allocated (as opposed to never launching at all which was their previous experience).
Its not that by being Lean and using Agile organisations are doing the wrong thing, but that in moving away from past processes we need to avoid throwing the baby out with the bathwater. Somewhere along the line we seemed to have forgotten that failing to prepare is preparing to fail.
Unfortunately, whilst the legacy tech remains unresolved, the leadership committed to propaganda, the budget processes analogue and we try to move fast without getting ready, we will continue to find ourselves in a web of wicked problems, many of our own making.
So what can be done?
The nature of wicked problems is that they are continuously evolving and resisting complete resolution. But that doesn’t mean we accept them and sit back. Like many things, the first step is to see them for what they are. Once you know what you’re dealing with, its then about making a series of affirmations that will guide you as you start to experiment with proper fixes.
Affirmation #1: Idealising for the end user isn’t valuable if you don’t take as much care to understand the contextual constraints you have to design within — even if those issues are not immediately apparent to the end user.
Business Analysis may feel a bit 90s compared to the cool world of being a Product Owner, but whatever you call it, you need to care enough to model the world you live in. It doesn’t matter how grand the future ambition is. In fact, the grander the ambition, the more important it is that you think carefully about taking the right first steps, and solving the right problems.
Affirmation #2: As a UX professional, it is no longer an option to have low technical literacy — start speaking dev, take time to learn about the big legacy challenges most organisations face and be pragmatic.
Let me be clear, I do not believe all designers should code. But I do find it odd when a designer does not have a strong mental model of the technological approaches that will be used to build the solution they design. I’d go so far as to say its irresponsible. Build a relationship with the development team. Communicate with each other through the medium of sketch, a storyboard works as well for mapping data through a tech stack as it does for a sequence of tasks. Take a Product Manager’s point of view and think about the whole, not just the bit you’re accountable for.
Affirmation #3: There is no blame game available to you. “They” is replaced by “We”. There is no right or wrong only this way or that way, based on the best understanding WE can achieve.
“They didn’t actually build my design”, “That’s not what we specced”, “It’s not my fault they didn’t do a good job”. You know designers who say this. You may have said this yourself as a designer. In a world where constraints evolve, releases are constant and priorities also change regularly, you can’t be precious about your plan or design. Wicked problems may persist, but their nature and associated symptoms change often. Each of us is invested in doing the best work we can, and the teams that get the most traction on addressing wicked problems, are the ones who are most cohesive.
Affirmation #4: Do not be beholden to the propaganda
This sounds flippant, but it is actually really hard. If your company is listed, then the financial press and media will look for the simplest and catchiest explanations of what you’re doing. Often, using the right language is about sending a signal to the market. But this cannot also be the way in which you do your detailed planning. Business leadership owe it to themselves, their colleagues, shareholders and customers to make better decisions. Get the right voices in your boardroom. Listen to your own teams as well as external perspectives. Look beyond the so-called accolades of experienced IT managers, and look for the hands on experiences that really tell the true story.
In conclusion,
If you’ve got this far, I hope it’s because what I’ve said resonated and you’ve self diagnosed some of the issues I talk about. Whether you agree with my phrasing of these issues as wicked problems or not, its important that as an industry, we think responsibly about how we create better digital transformation cultures. Not just to better serve the all important end-user, but to also hold our own heads up and say we did the best we could with the tools and information we had.
