The invisible threat of complexity
In product development there are certain considerations that will routinely need to be made when creating or changing products: How much does this change cost? What is the effort required for this update? How fast can we deliver this set of requirements? These questions are the bread and butter of a product manager. Yet there is a common factor that affects the answers to all these questions: the underlying complexity of the product or system at hand.
Complexity tends to breed more complexity, as changes in complex systems are complex themselves. Think of it as a vicious circle that once it begins is very hard to reverse, as there is a debt to be paid to undo complexity. A debt that only increases with every change that does not specifically serve to simplify things.
Complexity in software products
Complexity in software products is created when the default problem-solving activity is to add functionality and can creep in:
- the user experience — the part of the product that customers see and interact with, thus making the product harder to understand and use
- the underlying system — this can be complexity on the code level or in the way that the different subsystems interact with each other, making the product harder to maintain and troubleshoot
- on a process level — for example in the processes to deploy the product, gather customer feedback or provide customer support
Using a physics analogy, think of complexity as the entropy of a system. Every change in a system (or product) that is not done specifically to reduce entropy (or complexity) only serves to increase it. Increased entropy makes a system unpredictable and hard to control.
Similarly, increased complexity makes for products that:
- are hard to maintain
- have longer go-to-market times
- often offer a poor user experience
Complexity makes for products that are hard to maintain
The real cost of complexity is paid in the mid to long term as complex products are harder to maintain. The time that was spent implementing “just one more” feature to reach a short term goal is usually a fraction of the time that is needed to maintain this feature in the long run.
This time does not only involve engineering time (i.e. actual time coding around the complexity of the system so that a change does not break it down). It involves other aspects too, like the time needed to troubleshoot issues and respond to customer queries.
Troubleshooting time tends to increase exponentially with increases in product complexity. Anyone involved in troubleshooting issues in a very complex product will know what I mean. They will remember the frustration of the people involved as they strive to understand what is going on and questioning (maybe loudly) why the system is ever so complex.
Excess complexity also affects the time needed to properly document and communicate how the system works. A system can be properly documented or it can be under-documented. Both states are problematic for a complex system. Let me explain.
Obviously, an under-documented complex system is a time bomb waiting to explode. But over-documenting complexity is not a silver bullet either. In this case there would be an enormous amount of documentation that stakeholders will need to read and understand. Guess what will happen when they will need to troubleshoot an issue or support a customer under time pressure: they probably won’t read or understand it (at least not in its entirety). And mistakes will be made. That will require more time down the line to fix. And so on.
Complex products usually ship later than sooner
Complexity breeds risk. And risk is the bane of all project plans.
Complexity induces risk in planning in two ways. First, it is a nightmare having too many moving pieces on the board and trying to work out the interdependencies between them when trying to create a launch plan. Second, having each separate item on the plan come with risk attached on the delivery date only makes things worse.
Consider the following dialog excerpts:
- Q: “Will development on this item be completed by the end of the week?”, A: “Not sure, as the code is very complex and the engineers are trying to cater for all the edge cases.”
- Q: “Will we be able to finish the requirements analysis in X days?”, A: “Not sure, we need more time because the system is complex and the business analyst needs to align with all the affected teams to make sure that we do not miss anything.”
Imagine trying to create a project plan with the above. Add to that the communication overhead that a complex system causes when trying to get everyone on the same page and you get the idea.
Adding complexity is a sure way to jeopardize your launch dates.
Complexity leads in poor user experience
Complexity in software products tends to surface on the user interface. Complex user interfaces require more effort to understand and use and amplify the possibility to make mistakes.
Steve Krug in his famous book “Don’t Make Me Think” advocates that the more you make people think when using a software program or web site, the more likely they are to go elsewhere to get the job done. I strongly believe that this is true.
A complex product that attempts to do too many things rather than a few things very well will result in users looking for simpler alternatives that cover their needs better. If the value proposition of a product is not crystal clear in the minds of the users they will often have a hard time justifying their investment in that product. There are numerous examples of products that have offered streamlined user experiences and have won over more complex alternatives (for example Apple’s iPod).
An over-complicated user experience will drive users away from your product. Especially if there are simpler alternatives available. Or even if there currently aren’t any, then you are presenting an excellent opportunity to your competitors to disrupt you by offering them.
Keeping things simple
The best preventive measure against complexity is to actively guard against it and try to keep things simple where possible.
Make simplicity a key design principle for your products, in the user experience, in the technical level, in its processes.
Whenever trying to come with a solution to solve a problem take a step back and ask: ‘what is the simplest solution to solve this’? It could be to remove a step in the user flow instead of adding one. Or change a process. Or even a complete redesign of a complex technical setup.
Opening your team’s minds on that possibility is the first step towards decreasing complexity.
Software systems are inherently complex and it is not realistic to expect that you will be always able to avoid complexity. However, unless you actively seek simplicity in your designs then you will almost always end in the opposite side of the spectrum.
Change your planning language
Build your roadmaps using outcomes and not features. If you organize your product building efforts using features then you are actually prescribing complexity. Think about it: when talking about features you typically use the verb “add” (i.e. add a feature). Constantly adding features to solve problems leads to feature overloaded products.
However if you build for outcomes then you effectively decouple the implementation (the feature) from the solution. Instead of planning for features to solve problems a better approach would be to examine the outcome that needs to be achieved and plot the simplest path to get there.
Reducing the complexity debt
Making sure that no excess complexity is added is one thing. Most mature systems however should have already accumulated a sizable amount of complexity debt. Time should be frequently reserved to examine the system and identify areas that could be further simplified.
This practice is akin to reserving time to reduce technical debt in code. However, in this case you would be reducing the complexity debt that has accumulated over time, in the form of streamlining the user experience, the existing product processes, the user interface, the infrastructure setup and so on.
This action can be part of a wider continuous improvement initiative that may exist in the company and not as a standalone effort. Attaching it to a more generally accepted and endorsed work stream will often enable easier justification for the effort spent on it, especially if the company’s focus is on short term results.
Final words
Dealing with complexity in product development is easier said than done. It is in the natural state of things that any change to a system only serves to make it more complex, much like the entropy analogy.
It would be naïve to hope to eliminate complexity altogether, as software products are complex systems by definition and tend to get more complex as they evolve and mature.
However, if you do not take a vigilant stance and guard against complexity, you will often end up in uncomfortable situations where it will be very hard to push the needle, ridden by complexity debt that needs to be paid in order to move things forward.
