Software Architecture
How I Start the Technical Design Process
With a blank slate in front of you, how do you turn a pile of requirements into a technical design?
Recently I was asked what my approach to a new technical design is. I realised I’d never written it down how I go from nothing to an initial technical design, and that it was probably worth doing so, as much for my benefit. So this is an attempt at that.
Agile Architectures
It is extremely unlikely that the first architecture diagram you draw matches what goes into production. The problem space you’re working in will evolve and shift and the influences upon your decisions will change as your understanding increases. So expect to be flexible, care nothing when hours of work becomes redundant, and accept each new challenge to the design, because this is how your design improves.
Your choices for products & frameworks should ideally keep this principle in mind for future flexibility. Initially design in detail only what’s needed for your initial implementation and keep a loose model of what might be needed in the future, so that you neither design yourself into a corner, nor limit the future evolution of your system.
Decision Making Strategies
I have another post on Tools for Decision Making, which I think are useful when approaching architecture choices. This covers principles, mission statements, definitions of success and decision logs.
These are important because you’ll end up creating guiding principles for your design, which will be helpful to yourself and others later on. Additionally, documenting important decisions is a gift to your future self and other developers when they wonder why something is the way it is.
Best Practices
Testing, deployment automation, continuous integration, loose coupling, team independence, etc… These, and more, are all best practices that you should be aware of and considering when doing technical design. These days most technologies embody some level of automated testing & deployment, but it’s worth considering how the domain boundaries you decide affect these processes. Bundling multiple technologies into a single domain might introduce strong dependencies for development and deployment that will slow the team down and add unnecessary frustration to their work.
Starting Material
Your first step is understanding the functionality at a high level, comparable to the core Epics that you might create if you’re using Agile.
Before doing a technical design you should have a collection of requirements, user journeys, process maps, or just notes from meetings covering those. As a visual thinker I start by building a mental model of what the solution should do when it’s complete. But not how it’ll do so. Leave the technical details out of this for now and focus on abstract functional components.
Ideally requirements will be prioritised. Personally I prefer the MoSCoW prioritisation method, where requirements are assigned labels for Must Have, Should Have, Could Have and Wishes/Won’t Have. These lets you judge what levels of effort should be used to meet the requirements. In general your focus should be on the Musts & Shoulds and if a Could or Wish would require a substantial amount of work to achieve then it might be wise to review with the stakeholders if it’s cost effective to ever implement.
Conway’s law states that organisations are bound to produce designs that mirror the structure of the company itself. The counterpoint to this is that a design that does not mirror the structure of the company will encounter problems when it is implemented.
How these abstract components turn out later will depend upon later decisions but for now try not to assign properties to them (e.g. Stripe as a payment provider, MSSQL as a data store or a React SPA).
My first step is usually to name these abstract components and draw a box around each of them.
Boxes & Lines (aka Architectural Diagrams)
I find diagrams really useful for my own understanding of processes & systems, and also to convey knowledge to others. During design or discussions I’m happiest drawing boxes and lines on paper or whiteboards and my finished articles are often just polished versions of those.
However beware of putting too much information into a diagram. Often diagrams are only meaningful to those involved in their creation, or advance knowledge of the process involved. To get around this I try to keep my diagrams simple, layered and segmented.
- Use different diagrams for different scopes. A single diagram should display a single level of understanding. e.g. one to display the context the system exists within; another to display the components of the system; and at one or more to cover the specifics of processes within each component.
- Display a single process or pathway in a diagram. Diagrams get very complicated when you overlay multiple processes on a single image, so limit what you put on there.
- Your diagrams should be easy to understand by someone who has no understanding of the complexities of the system other than the context it exists within. A sequence of diagrams should be used to convey that context if needed.
A great introduction to good diagramming is the C4 Model from Simon Brown and I highly recommend adopting a similar approach for your work.
Establish Boundaries & Communication Patterns
How that you’ve got some building blocks to play with you can start shuffling them around and thinking about domain boundaries, functional alignments and communication patterns.
Even an architecture of moderate size is likely to support having independently developed and deployed components, be it a CMS, front end or API. Think about what makes sense to group and what can be an independent domain. Find the middle ground between everything existing in a monolith, and a very distributed big ball of mud.
These independent systems need to interact & synchronise somehow, be it via a data hub , API or centralised controller. These lines between boxes will later become documented contracts for how your components interact but for now keep it simple and do a proof of concept only if you need to validate an untested process.
For cross domain communication think about considerations like speed, synchronicity, logging, reproducibility, etc… to chose the right method. For example if your downstream order processes take a long time then the communication towards them should be one way and not a dependency of real time user actions.
Product, Framework & Technology Choices
When considering technologies and products there are a number of factors that you need to consider. In general, avoid a choice just because it’s the latest and greatest thing which everybody is talking about. Each choice should be justifiable for one or more reasons. Also avoid “common sense” decisions, as common sense is merely your current opinion as influenced by outside sources that align with your existing preferences. That is to say, it’s not necessarily the correct choice for this problem.
Considerations include:
- Practically — This covers a lot, such as what is the existing knowledge of technologies within the team; what’s easy to develop and deploy using best practises; is there a good community and available knowledge to tap into; etc…
- Cost — It’s unlikely you’ve got an unlimited budget so you need to keep one eye on cost. Is that incredibly expensive CMS really going to be used to its full potential? Are there cheaper options than using a database with expensive reserved units when you only need a light load? If you do get an unexpected burst of users what will happen to the cloud costs?
- Functionality — Is about the right tool for the job. Not over-engineering the solution, neither under-powering it. Find the balance.
Pros / Cons / Mitigations
When working through options a simple list of pros & cons can be really helpful. I like to add a couple of extra fields:
- Effort — an arbitrary number, usually between one to 5, of how difficult the option is to implement. I’ve used this in the past to work out not only product choices but also the correct phasing of projects. Sometimes there is a cumulative effect of changing the order of phases which affect the work needed to implement each individual phase.
- Mitigations — There might be a downside to an option which a sensibly sized action or solution reduces or negates. These mitigations allow you to sensibly judge the risk of each option. For example, if nobody in your team knows a technology can you send them on appropriate training and/or hire a short term expert to support the team?
Phasing
Building a MVP of your product and technical architecture is the best way to validate it’s design. Plus it brings benefits of getting real world feedback and quicker implementation time for your customers and your business. Where possible identify, along with the business, what processes should/could be introduced first and structure a phased development plan around that.
If you do prioritise an MVP then focus on the technical design needed for that initial phase and leave the latter phases as higher level drafts. Your design might well change once it hits real world scenarios, so best not to commit too much time to these latter phases where everything might change, or not be needed. As above, this is about keeping your architecture agile, evolutionary and “just-in-time” design decisions.
Similarly build for a realistic scale. If your online product will only have about 100 simultaneous users then you don’t need to build a self-scaling, memcahced solution that can burst to a million simultaneous users on day one! Sure it’s useful to build components that can scale later on, but avoid the extra complexity that scale brings until you know that it is actually needed.
Rinse and Repeat
The first version of a technical architecture is unlikely to be your last. The design needs to be researched, tested, discussed and perhaps proof of concepts created to validate assumptions and processes.
Software architecture should never be about handing down a design completed by a single individual to a development team. Such designs fail as they don’t have buy in and feedback from the developers, so structure discussions and feedback into your design process.
Lastly, evolve your architecture as much as possible as early as possible. The earlier a change is made the cheaper it is to make, so take the risks early rather than find out a substantial change is needed once it’s in production, because nobody needs that!
