As an architect, we are constantly trying to design the perfect system; something that is sustainable, meets the future requirements, minimizes technical debt, leverages all the right tools, standards. It’s a masterpiece that will live on well beyond our time on the project. That is what we hope.
In reality, while many architects have designed these systems, it is the exception that make it into production. More likely the architectures wind up thrown aside by development, not invested in by product owners or only partially implemented before the business can no longer wait, resulting in a less capable but easily available solution to be put in place and your design is thrown aside.
I can attest to having many of my architectures never see the light of day. They’re just diagrams living in some corporate cloud to confuse a future architect or developer of what systems exist within the network.
Of course, there are ways to avoid this cycle. Ways that meet the goals of the architect while increasing the buy-in of the product owners and improving the time to market.
Use Existing Solutions
The first recommendation is the most obvious: reduce your cost and time-to-market by building around existing open source or cloud solutions rather than build a custom solution. While this seems obvious, in practice many obstacles are barriers to such solutions.
- Cloud solutions are not fully adopted within the organization
- Devops limits the cloud technologies that can be used
- Corporate or government compliance for using new 3rd party software is higher or more complicated than for using internally built code
- Developers fall back to languages or tools they know rather than embracing the unknown
- The organization perceives more risk in using something new than the cost of building it themselves
- The organization perceives more risk to the 3rd party solution not being properly maintained without properly considering the internal maintenance tail
- The cost of existing developers is baked in while buying cloud or 3rd party capabilities requires allocating new funds
In multiple organizations, I have found my role to be breaking down these barriers for developers or analysts, enabling better architectures rather than directly designing the systems themselves.
Compare Fairly
A common misstep occurs when architects or developers compare their fully created (non-existent) end architecture against existing solutions. If you are comparing your future 2 year out solution, make sure to take the following considerations in as well:
- Where will the existing solutions be in 2 years?
- What is the cost and risk of building your own solution?
- What is the opportunity loss of not releasing earlier while building your custom solution?
- What is the opportunity loss of not allocating the company time and developers to other capabilities instead of the custom solution?
Sometimes architects or developers will identify a requirement that existing approaches do not meet and use that to justify building something new without first providing the alternative option to a product manager. For example, is this requirement worth an additional 2M in development and a 10-month delay in schedule, or can you do without that missing capability? Properly considering these options is one reason why an architect exists. Sometimes it’s not about the architect managing and solutioning the complex or detailed system but instead about helping to avoid needing the complex solution all-together.
Design for the long term, Build for the short term
If it is necessary to build new, then design for the long-term but build for the short-term. Here’s how: build an initial scaled down version that meets immediate requirements, but make sure to identify the components needed to meet the long-term objectives of the system.
To do this, first envision an end-architecture to satisfy the long-term goals in a traditional system architecture design document. Then, develop a simplified version of this architecture that includes only the capabilities required for the initial release. This is one of the most difficult parts, because it means throwing away most of what you learned and decided in your initial architecture document. The first document is used to identify the components and capabilities required to meet the long-term objectives of the system to facilitate making strategic decisions in the initial scaled down version. The key thing to remember is that those extra capabilities are not currently required. When they are, you’ll have the opportunity to build them, and you may have even evolved your designs into something better by then. If the capability is never built, that (should) mean it was never required and you just saved the organization from building something unnecessary.
Move quickly, but invest intelligently
So where do we invest? Here are some questions to ask:
- What components are interacting with the user or other parts of the system in the initial release?
- What components are doing the core processing required for the initial release?
- What capabilities can be removed, replaced, or simplified with something else to meet the initial requirements. Replace away!
After following these steps, we will have identified the core capabilities that need to be built. We can then define integration contracts around these core components while considering our future architecture. This is where we invest to protect against rewriting these components when new requirements arise.
We are not changing the architecture; we are changing the order in which the components of the architecture are built. What is the minimum set of components necessary to meet the first set of requirements and are there unnecessary functionality of those components that can be reduced as well?
Are we investing in some capabilities that will need to be ripped out to meet the long-term objective? Yes! but we have also taken months, perhaps years off the schedule to meet our initial requirements. As PM requests new capabilities, we may need to rebuild or rip out portions of our infrastructure, but those investments will be made in line with the capabilities they offer rather than an upfront cost that will not be realized until a potential future state. Since we are conscious of the system’s long-term architecture though, we can make guided choices that will continue to evolve the system towards the end goal.
Occasionally, it is useful to reconsider that end goal. Is the end architecture originally created still valid? If the answer is yes, I’d consider taking a harder look or having someone less attached provide the critique. Technologies will change, requirements will change, and most importantly your knowledge of the problem and its solutions will have changed as the initial versions of the system have been built.
My team likes to talk about full architectures as a north star. I would agree with that categorization. A north star is there to guide you, but if you attempt to sail all the way to that star you’re probably (just guessing) not going to make it.
