Cloud Adoption and Cloud Consumption Principles (Google Cloud Adoption Series: Part 3)
Welcome to the continuation of the Google Cloud Adoption for the Enterprise, From Strategy to Operation series.
In the previous part, we talked about how to establish our Cloud Adoption Business Case. In this part, we’ll talk about the importance and value of establishing a set of core Cloud Principles.
What Are Principles?
Let’s start with a dictionary definition:
I like these…
- “A fundamental truth or proposition that serves as a foundation for a system of belief or behaviour, or for a chain of reasoning.”
- “A rule or belief governing one’s behaviour.”
- “A natural low forming the basis for the construction of a working machine.”
- “A fundamental quality determining the nature of something”.
Let’s bring it slightly closer to home. TOGAF (The Open Group Architecture Framework) defines architecture principles as follows:
“Principles are general rules and guidelines, intended to be enduring and seldom amended, that inform and support the way in which an organization sets about fulfilling its mission.”
Why Do We Need Them?
I’ve had people tell me that they don’t like architecture principles. That they’re just hand-wavey architecture mumjo jumbo. I disagree strongly with this viewpoint. For me, they are absolutely crucial in providing a basis for efficient, conscious, consistent, repeatable decision making, supported by reusable rationale.
When selecting a new technology for use in your organisation, or selecting from a set of possible technologies within a particular solution, we should always ask ourselves: “Does this decision align to our architecture principles?” If it doesn’t, we should question whether the decision is sound.
So having a set of agreed principles brings a number of benefits, which I’ll now cover.
Alignment to Strategic Goals
Our agreed set of principles should be pre-aligned to our strategic goals. And consequently, when a new solution or decision is made that aligns with these principles, we can be reasonably sure that this decision is not opposed to our strategic direction. In short, such a decision will not be detrimental to our strategy.
Consistency Across the Domain / Organisation
Our agreed set of principles helps our distributed organisation to make consistent choices. It helps the organisation move forward with a form of convergent evolution, where the principles act as a set of constraints or guardrails, funnelling different projects and teams to come up with consistent answers to common problems.
The principles help different teams understand what design choices are allowed, and also helps to filter out those choices that are not allowed. In doing so, we help to ensure a smaller set of technologies are used in a given domain. This means we get:
- A smaller product set.
- More technology reuse.
- A stronger core set of reusable skills.
- Improved cost efficiency.
Delegating Design and Decision Making
If we properly share, socialise, champion and embed our set of agreed principles, then we can safely delegate responsibility of architectural decision making. We can ask our architects and senior engineers to always validate their decisions against these core principles. And that way, the quality of decisions and designs will typically be much higher than it would have been, by the time the solution lands in front of some sort of architectural governance group.
More Efficient Documentation
One of my most popular articles on Medium is: Architecture Documentation — Where to Draw the Line. In that article, I make the following statement about one of the key motivations for creating architecture documentation:
“… it captures the rationale for every decision that was made. Having this rationale is crucial. Without it, decisions will be revisited again and again, wasting time and resources. And worse, without the rationale, a decision may be reversed later, without fully understanding the implications. This could be very costly!”
And so, all good architecture documents should record a set of decisions that are relevant to the architecturally significant requirements. And all of those decisions should include rationale. Having agreed principles can really help here, because in many cases, alignment to one or more principles is the rationale! You can often avoid writing a load of extensive (and repetetive) rationale by instead pointing to the principles this decision supports.
Streamlining of Architecture Governance
This is a big one and should not be underestimated!!
Architecture governance is an absolutely necessary evil. It is required to ensure that:
- Individuals are held accountable.
- Solutions conform to strategy, policies and principles.
- Appropriate stakeholders have been engaged and consulted.
- Rationale has been properly captured and is understood.
- Trade-offs and risks have been properly managed.
- Technical debt is continuously managed.
Some organisations do this badly. Some organisations make a cottage industry out of architecture governance. Have you been in an organisation where the same decision or solution ends up being presented at three or more separate architecture governance authorities?
Here’s where the principles really help:
- If an architectural decision, technology selection or design fully aligns to your organisation’s strategy, policies and principles, then there is no need for heavy scrutiny. I like to refer to the governance required at this stage as “governance-lite.”
- On the other hand, if the decision, technology selection or design does not align, then expect heavy scrutiny. I.e. expect to have to present the decision at a architecture review board or solution design authority. And expect to be asked questions like:
— “Why have you selected x, rather than y?”
— “What is the rationale for selecting this technology?”
— “Have you considered implications a, b, and c?”
— “Is this a tactical decision, or a long-term decision?”
— “Does this need to be recorded as technical debt?”
— “Do you have a plan for how you will pay-off this technical debt?”
There may be good and valid answers to these questions.
In short: expect decisions aligned to principles to be supported, and to be approved quickly. Otherwise, expect to be challenged.
But if you don’t have a set of principles, then it’s much harder to apply the governance-lite approach.
How to Articulate Architecture Principles
Every organisation should create their own set of architecture principles, which are agreed, published, and embedded. That being said, you don’t need to reinvent the wheel! There’s only so many good principles, and if you like some principles you’ve seen at another organisation, then copy them!
When it comes to actually recording and documenting your principles, I like to use the TOGAF approach. In short: every architecture principle should contain these four components:
- Name — A memorable short name (typically a phrase) that captures the essence of the principle.
- Statement — Succinctly and unambiguously articulate the principle.
- Rationale — Describe why this principle should be adhered to. Capture any details that will help a reader understand how to adhere to the principle. What are the intended consequences of doing this? What are the expected consequences of NOT doing this?
- Implications — Describe the anticipated additional consequences of this principle, aside from the intended consequences described in the rationale. I.e. additional practical impacts, planning required, or “gotchas” that we need to be mindful of.
Some tips:
- Make sure you engage a wide group of stakeholders in building your principles. You want buy-in. If you don’t engage your stakeholders early, you’ll meet more resistance later. Even if your principles are great!
- Make sure you formally approve your set of principles. They need to carry authority. Otherwise they’ll just be ignored. Typically, approval would come from a suitable architectural authority, such as an Architecture Review Board.
- There are many ways you could document these principles. You could have a page for each — e.g. in a site, or in a Confluence page. You could master them in a spreadsheet. Wherever you master them, it’s crucial that they are published, easily discovered, easily consumed, and well known.
Granularity of Principles
Architecture principles are fractal. What I mean by this: organisations often define architecture principles at different layers of granularity. For example, it is common (but certainly not required) to have these layers:
- Enterprise Architecture Principles — these are organisation-wide, and align to the overall business strategy.
- Technology Principles — a set of principles that guide the use of the technology stack.
- Data Architecture Principles — these guide how to organise, safeguard and democratise the use of data.
Since this series is about how to adopt cloud in the enterprise, I’ll now give examples of Cloud Principles, which would typically be a subset of the Technology Principles from the list above.
Principles for Use of Cloud
Here I’ll document a number of principles that I like to use. Don’t be surprised if you’ve seen them elsewhere. For example, some are lifted from Google.
Furthermore, if you were to list the technology principles from a group of technology organisations that do IT well, you would expect quite a bit of commonality and overlap. (Though there may be more variation when looking at the Rationale and Implications, because these somewhat depend on your organisation’s current maturity and capability.)
Below is my recommended set of Cloud Principles. If you follow these principles, you’ll extract the most value from cloud, and minimise your risk. If you don’t follow them, then you will eliminate much of the benefit of using cloud.
- Host in public cloud
- Use managed services
- Use cloud-native services
- Avoid commercially licensed software
- Automate deployments and installs
- Immutable infrastructure (“cattle, not kittens”)
- Document everything
- Manage lock-in
- Systemic FinOps
Now let’s look at them in a bit more detail.
Host in Public Cloud
Statement: Deploy workloads to public cloud.
Rationale: Here, you could simply list all the benefits of public cloud. I.e. the types of benefits we covered in the strategy and business case sections of this series.
Implications:
- Organisational capability implications. E.g. training required.
- Impact on asset and license management.
- Op model, particularly around DevOps, support, SRE.
Use Managed Services
Statement: Use cloud provider-managed (e.g. Google-managed) services, and avoid self-managing / DIY. E.g. managed database (such as CloudSQL) in preference to installing DBs on a VM.
Rationale:
- Reduces operational overhead. (E.g. no OS management, software installs, patching, upgrades.)
- Increases agility of deployment.
- More easily automated.
- Easier to configure for HA, DR, backups, etc.
- Improved security posture.
- Underpinning technologies typically do not have a commercial license.
- Often relatively easy to migrate to, for existing on-prem workloads.
Implications:
- Organisation changes, e.g. reduced need for OS engineers, DBAs etc. Consider retraining opportunities, or cross-training as cloud platform engineers.
- The need to explain the change of op model to application dev teams.
Use Cloud-Native Services
Statement: Adopt cloud-native services for new workloads, where possible. For migrating workloads, evaluate the ROI of leveraging cloud-native, and decide accordingly.
Rationale:
- Increased agility.
- Greatly reduced ops overhead. (No servers to manage.)
- Loosely coupled services allowing independent development and independent deployment.
- Fast and easy scaling. Often with scale-to-0.
- Fault-tolerant by design.
- Built-in security and compliance.
Implications:
- Migrating applications will likely need rearchitecting to use native services.
- Potentially (but not necessarily) increased vendor (e.g. Google Cloud) lock-in.
Automate Deployments and Installs
Statement: Automate infrastructure deployment using infrastructure-as-code (IaC) from the start, and throughout.
Rationale:
- Fast deployment of infrastructure. (Minutes, rather than days or weeks.)
- Infrastructure deployment is now part of the CI/CD pipeline.
- Enables rapid changes, and rapid rollbacks.
- Repeatable, consistent deployments.
- Eliminates configuration drift, versus the design.
- Eliminates configuration drift between environments.
- Is self-documenting and version controlled, eliminating significant effort in writing LLDs.
- Enables “cattle, not kittens”.
Implications:
- Up-skill on IaC technologies. (E.g. Terraform.)
Immutable Infrastructure (“Cattle, not Kittens”)
Statement: Resources should be treated as redeployable, interchangeable entities. Once built with automation, a resource should not be modified. If a resource must be modified, it should be redeployed.
Rationale:
- Automation through IaC means that resources can be reprovisioned in seconds. It is actually easier — and more consistent — to modify the IaC and redeploy, than it is to modify something that has already been built.
- Redeploying through IaC ensures consistency, and that the change is documented and repeatable. Conversely, modifying a resource outside of the IaC results in often unmanaged configuration drift.
- Scalability — we can deploy and manage at scale. (Think: “cattle”.)
- Reduced errors — because we do not allow unmanaged, human error.
- Efficient recovery from failures — because we simply deploy new instances of a resource. (E.g. VMs, containers.)
Implications:
- It is crucial to embed an SRE culture that properly implements SLOs, monitoring, health checks, alerting.
(FYI, this concept of “cattle, not kittens” — aka “cattle, not pets” dates all the way back to 2012! The concept is that kittens need looking after. Consider a legacy on-prem application, where every VM has a name, and every VM needs lots of maintenance to keep it alive.)
Document Everything
Statement: Ensure the appropriate and proper set of documentation for all systems and solutions. E.g. architectural documents, operational run books, etc.
Rationale:
- Knowledge capture.
- Education and onboarding.
- Rationale for decisions made.
- To facilitate support and forensics.
- (See more in my article: Architecture Documentation: Where to Draw the Line?)
Implications:
- Training on how to create appropriate documentation.
- Appropriate knowledge management / doc management tooling.
Manage Lock-In
Statement: Consider and mitigate vendor lock-in risks whilst leveraging provider-managed and cloud-native capabilities.
Rationale:
- Managed and native capabilities bring significant value, and are key to leveraging much of the value of cloud. However, these technologies come with a degree of lock-in that we must be mindful of.
- We should weigh up — where appropriate — how we might alternatively run these workloads, if forced to exit from our chosen cloud provider.
- Broadly, when using managed services that are underpinned by popular open source technologies, the lock-in risk is low.
Implications:
- Continuous evaluation — at cloud strategy / cloud governance level — of our set of endorsed and curated cloud technologies, including assessment of exit approach.
- Contractual agility.
Systemic FinOps
Statement: Embed Financial Operations (FinOps) practices systemically to monitor, manage, and optimize cloud costs through data-driven decisions, promoting visibility and accountability across all stakeholders.
Rationale:
- Cost optimisation — continuous assessment and continuous optimisation, to reduce costs and ensure efficient resource utilisation.
- Data-driven decisions: using actual resource utilisation data.
- Accountability: when business/product/service owners have visibility of their own costs, they are more motivated to optimise those costs.
Implications:
- Decentralisation of financial management — everyone should care.
- Shift from capex to opex may need changes to the organisation’s financial operating model.
- Appropriate tooling deployed, to ensure cost transparency and visibility.
- Establish a FinOps function to coordinate the distributed FinOps.
Wrap-Up
I hope this has helped you understand the importance of architecture principles. And I hope my initial set of Cloud Principles is something that might help you, as you build your own.
Next, I’m going to get into some fun stuff: mapping the as-is to the to-be technologies.
Before You Go
- Please share this with anyone that you think will be interested. It might help them, and it really helps me!
- Feel free to leave a comment 💬.
- Follow and subscribe, so you don’t miss my content. Go to my Profile Page, and click on these icons:
