Why is writing software so complicated?

And what can you do to limit complexity?

Complexity in domain

Every business task has an inherit complexity. A simple application that can display images is less complex than a financial application that makes predictions in stock markets.

This means that whatever is being worked on, there is necessarily some complexity.

It’s worth noting that it is the interactions of people that bring complexity into designs. If there are multiple people trying to achieve something different by using software, then it will need to be more complex in order to cope with the scenarios different users will throw at it.

For this reason, it can be beneficial to split up applications into small web services with specific aims, used by a subset of all users. This separation of concerns allows developers to concentrate on one user base, and not be lead astray by concerns about details from other application areas.

The user base is not the only people aspect that adds complexity to an application. The business itself can also add complexity.

Businesses often know they want something, but are not completely aware of what that is. This results in changes of requirement during (or towards the end of) the development process.

Complexity in tech stack

Different languages add different sets of complexity to projects.

There are extremes to this. Scratch is very simple to use, but is therefore also limited in how you can use it. Brainfuck is very complicated to use, but can be used to write more extensive projects (but I wouldn’t advise using it). Here is a brainfuck compiler written in brainfuck.

When choosing a tech stack to use, there are pros and cons in what you choose. The most important factors are:

  • Your teams existing knowledge
  • The application’s business logic
  • The expected size and scale of the application

As an example, Angular is a very good Javascript web framework. However, it does a lot automatically behind the scenes to wire components together. This means there are patterns which must be followed when using it. If the patterns are not used, programs either will not work, or will become very complex very quickly. Conversely, React, another very good Javascript framework, is not as prescriptive in how you use it.

This means that a team that knows Angular already will write bigger apps more quickly using Angular. A team that knows neither will initially write apps more quickly in React. A longer term project would likely benefit from the team spending time upfront learning Angular, and therefore becoming more productive in the long run.

This example considers the pros and cons of two frameworks. There is a further decision to be made about which language (or combination of languages) to use. For example, there can be performance benefits to writing code in C, versus a higher level language like Ruby. These come with the downside of lower level languages being more complex. For this reason, the purpose of the application is an important factor when choosing technology.

Overall, it’s clear that the technology chosen to write an application is going to impact how complex it becomes in the long run.

Complexity in programming style

The final element that will add complexity to any programming application is the developers themselves.

There is a lot of advice (created by insightful people over years of trial and error) about how best to write programs. YAGNI, SOLID, etc. are all important principles, but it is also important to know when your application does not fit into these patterns, and the rules should be broken. Without this hard earned experience, programmers will write code that:

  • introduces bugs
  • is more difficult (and costly) to extend later on

There are several ways to reduce technical debt on projects.

Writing tests (at all levels), will encourage good practices. Where it is difficult to write tests, it implies that there are already problems with the code, and therefore writing tests can be a good way to find these problem areas and fix them (in a safe way).

Tests are also a good way to prevent against regressions. This is especially important in larger projects where there are multiple teams working on the same code base increasing the likelihood of nasty merge conflicts.

Static Analysis is a second (but not as effective) method of finding poorly written areas of the code. Static analysis tools can highlight (at a low level) areas of the code that have been written poorly, along with advice on how to improve the code. It is important not to blindly follow all the advice though, there will certainly be times when static analysis tools give bad advice, that can lead to defects.

Monitoring files for churn is a third way to monitor for problem areas of code within an application. This is a more tricky thing to monitor, but sensible scripts written against version control can highlight this. The reason this indicates complexity in code, is that a file which needs to be updated regularly clearly has lots of responsibilities and is therefore more complex through the interaction of these different functionalities.

In order to reduce churn on files, consider separating the functionality of files out into separate files, which can work together in order to perform the functionality of the original larger class.

Your aim as a software engineer

From the above, it’s clear that things are always going to be complex, but there are certain things that can be done to reduce this complexity.

  • Aid the business in choosing it’s aims (based on how complex they will be to implement vs how much value they will add to the company).
  • Ensure that the correct tech stack has been chosen for the task that is being completed. If it becomes clear that there are better options later on, consider moving to them, but also consider the cost of moving.
  • Make sure developers are trained in best practices and keep up to date with the industry.
  • Make sure tools that exist to aid programming are being used properly (unit tests, static analysis, etc.)

The above goals should be performed with the aim of reducing the complexity of the project as much as possible, with the lower achievable limit, being that of the business logic that is being developed.