Solving the problem of complexity in software
With the constant proliferation of new platforms, frameworks and languages, it can be difficult for developers and CTO’s to keep up, and also to work out which they should be learning and using, and which they should be avoiding.
The ongoing addition of new features and abilities to programming languages makes software more complex too. Add to that the need for continuous integration and deployment, and the opportunities/challenges of Cloud infrastructure, and it’s a difficult world to navigate safely and confidently. But there are proven ways to avoid many of the problems of growing complexity that I’ve described.
I’m CTO and co-founder of two software companies — Remote and Code Assembly. Remote was started in 1999, and we enable medium-sized businesses to run more efficiently and to scale up safely by automating their delivery processes with custom software. We’ve also built large applications and mobile apps for global companies like Volvo, Sony, and Volkswagen Group. Code Assembly was formed this year, and our first product, incoder, is a revolutionary code generation platform for software developers.
The complexity problem brings about two main difficulties.
Firstly, it’s very rare that an application will consist of solely one language or framework these days. It’s much more likely that it will be built up of micro services that connect together with API’s, and those micro-services may well be developed by different engineers, in different languages. Maintaining consistency and stability between these ever growing frameworks is a challenge, especially if a new framework update needs us to amend syntax or refactor in some way.
Secondly, as there is never one way of achieving a solution to a problem, and there are so many patterns that a developer could use, inconsistency with an application can cause real issues when maintaining and refactoring work.
At Remote, we’ve discovered several disciplines that reduce the potential for disruption.
1. Consistent patterns and models. Design a strong model that uses patterns of enterprise application architecture that are easy to read, good to scale, and flexible enough for use throughout the application. Insist that the whole team stick to this pattern/model as much as possible, so that they are very familiar with it. This way, there’s consistency across the whole application, and developers can immediately know what a piece of code is doing even if they didn’t write it in the first place. At Remote, we love using the Mediator pattern, for example.
Also, choose a solid, reliable and well-supported tech stack, and stick with it. That way, you get to know it’s features and the best ways of working with it. If you change to a different stack for every project, it’s difficult to get your head back into the specifics of each stack as you switch between them. At Remote, we’ve settled with React, .NET Core, Entity Framework, and SQL Server — our work has been of consistently higher quality since we chose the stack and stuck with it.
2. Domain Driven Design. As an application grows, and new functionality is requested, it often makes sense to add and expand on existing methods or classes to accommodate the functionality. The risk here is in ever-growing methods that are used, expanded on and reused again throughout an application. This results in single points of failure, and the inevitable scenario where a fix in one area of the application breaks something in a different part of the same application.
Domain Driven Design means that you can separate areas of your application into sensible domains that are based on the business itself. This makes the code more readable, and easier to manage. The separation of concerns also means that you’re less likely to break something while ‘improving’ something else.
As our lead developer Jon Farrar says, “Complex problems are just lots of simple problems combined, but you can’t see the wood for the trees with large applications until you separate everything out”. Domain Driven Design is the way to do this.
3. Refactoring. When an urgent support request comes in, or you’re close to a deadline, it can be tempting to amend existing code or ‘hack’ a fix in place so that you can ship early. The problem with this is that once you’ve shipped, it’s rare that you’ll go back and refactor the code so that it follows the correct pattern and maintains consistency with the rest of the application. A year and a hundred hacks later, and the code is a mess; difficult to read, badly formed and hard to debug.
Once your code’s a mess, what could have been fixed in half an hour might take a day or more while you unravel it.
The vaccine for this problem is ongoing refactoring. Take time to adjust, simplify and reformat your code whenever you visit it to make it readable and consistent with the pattern of the main application. Although it may take a tiny bit longer to deliver your code, you’re investing in future delivery.
The motto ‘leave the code tidier than it was when you found it’ works well here, and it brings huge longevity to the application, and turns your future days back into hours again, and prevents your code from becoming too complex to work with.
4. Testing. To aid your refactoring, you need a good safety harness and warning mechanism to make sure that as you move code around, it still works as it should. Unit tests are another great investment in time up-front and speed up both your development and your bug fixing process in a big way. Most importantly, as your application gets bigger and its complexity grows, the unit tests alert you to issues without you having to test the same code over and over again.
Unit tests are not just great for helping with refactoring — Test Driven Development (TDD) keeps your programming well formed, and and simple as possible. The concept is that you write a test for each tiny function, and then write the code to pass the test. The way that your code is formed as a result of writing the tests makes it clear and easy to use, and being driven by the tests means that you have be absolutely sure about your intended outcome before you start programming, which keeps things as simple as they can be.
Alongside unit tests, end-to-end testing frameworks like Cypress have dramatically improved the quality of our code, and equally reduced the time it takes to test before we deploy.
As great and essential as automated testing is, an actual human checking that everything is as it should be is essential. We have an internal testing phase of our deployment process, which means that nothing is deployed without being tested by a different member of the team (who didn’t write the code).
Lastly, we don’t deploy directly to production; code is deployed to a UAT (User Acceptance Testing) environment which mirrors the production environment, so that our clients can go through the new work themselves to sign it off before we go live.
5. Code generation. If your technical leads and CTO’s spend time upfront designing a strong model for the application, creating a series of templates to generate the code based on the model is a powerful way to create consistency across the application, and it also speeds up the development process tremendously.
Our new software, incoder — due for release in the new year — allows technical leads and developers to declaratively describe the model, using domain driven design to separate the different aspects of the domain, so that the code it produces is elegant, easily readable and easy to maintain.
Following these disciplines may be difficult to get into the habit of at first, but it’s well worth the diligence of doing so, as it means that you can build large, scalable applications without them becoming too complex and difficult to maintain.
Originally published at https://remote.online.
