Software Principle #1: Earn Your Complexity

Complexity is everywhere. In every domain, under ever rock, lies something that requires years of experience to fully understand.

In software development, we learn about these complexities and try to model them using computer languages in our best guess efforts. The thing is though: very rarely are engineers domain experts.

After all, we studied how to model data, design elegant algorithms, build efficient systems and deliver those to production. Nowhere in there was an in-depth study in medical sciences, the legal system, or any of the domains we’re asked to build systems in every day.

This alone presents a problem: it’s tough for a non-expert to see how a complex system/idea fits into a specialized ecosystem.

As a profession, we’ve started using product managers who are experts in breaking down problems, figuring out the details at a high level, and delivering these details in technical terms to the development team.

Sometimes, the product managers come from the domain they are breaking down and have intimate knowledge of the domain they are executing in, but often they don’t: they come from a development background. This source of complexity causes whole systems not to ship, and sadly it’s not the only source.

The secondary source of complexity comes from the systems themselves. Designing computer systems to solve challenging problems is a daunting task.

Delivering applications at “web scale” requires expertise in a vast number of categories:

• Caching (what to cache, how to cache, how to refresh your cache, which cache records should be invalidated, when should they be invalidated, etc.)
• Database tradeoffs (how normalized should my data be, how normalized can it be, should I use a document database or a relational one, if I use a relational database, what database engine should I use if I use a document database what am I optimizing for, etc.)
• Architecture (micro-services or monolith, build vs. buy, boundaries in the system, etc.)
• Infrastructure (how should my network work, what is my replication plan, should I geo-replicate for redundancy/latency, etc.)
• Code optimization (readability of the code vs. efficiency in the algorithm)

This list scratches the surface and just for a web-scale application, but what about IoT devices? In some ways they are even more complicated as they combine web-scale technology and hardware.

Even then, underneath all the technology we use to solve these problems are thousands of layers of complexity in themselves.

For example, consider a Kubernetes deployment in the cloud: more than likely, the servers used to create the Kubernetes cluster are using virtual machines which sit on top of a hypervisor which may or may not be sitting on an operating system (these days there are “bare metal” hypervisors).

We then install Kubernetes onto those virtual machines and then when we deploy our docker containers into Kubernetes. We now have 4 layers (hypervisor, virtual machine, Kubernetes, docker) and then we have our already complex applications.

All this complexity is easily obtained just by starting a software project and therein lies the problem: complexity creates problems. Defects are created when the solution is implemented with imperfect knowledge. The more complex a topic is, the more likely a person has imperfect knowledge.

Imperfect knowledge is where principle #1 comes into play: Earn your complexity.

Keep It Simple

You must always seek the most straightforward solution possible for any given problem. and understand things must be able to evolve as you gain more knowledge.

It is incredibly easy to add something to a simple system but can be nearly impossible to add something to a complex system.

Poor architectures and systems are built when people don’t earn their complexities. The industry is rife with developers using more components than they need, adopting tech that isn’t required, and writing overly complicated solutions for problems.

All this is done while trying to solve a problem in a domain that they don’t fully understand. You must earn your complexities over time. The more complexity in any given system extends the amount a developer must understand to make a change; whether that is for support or extension of the system.

It’s important to note: you can’t avoid complexity. If you solve any problem with sufficient depth, it creates complexity. Great architects and developers know where they can absorb complexity as their teams, and themselves become more mature over time. The classic example of this is micro-services.

While the systems within a micro-service architecture are incredibly simple, thus the moniker of micro, the overall systems that are required to support such a distributed system are incredibly intense. The architecture didn’t eliminate complexity: it moved it to the operations of the system itself.

As the maturity of developer operation tools increases at a faster and faster speed, it helps but not remove the complexity altogether. There are no silver bullets here: just lead. You must know where your complexities lie because 9 times out of 10, that is where your bugs, system outages and other issues occur in your system.

Making informed decisions regarding complexity and deferring it until you can handle it creates cleaner systems that are more maintainable and extendible.

Also, it allows your system to eventually adopt the needed complexity as you work in your domain. Just like companies can’t be everything to everyone, neither can a computer system. Large domains like the medical or legal domain are too large to model into one system, and even if you managed to do it, the system would be untenable for anyone who is not an expert in the field.

Our job as engineers is to take complexity, minimize it and implement simplicity. There is no great art in building a complex system, but there is in building incredibly simple ones that solve complex problems.

Andrew Wolfe, CEO Skiplist
Originally posted on https://www.skiplist.com/