Learning more about FinOps on Google Cloud
In this article, I follow up on my previous post Introduction to FinOps on Google Cloud with a list of useful resources to learn more. FinOps is becoming more important and relevant with cloud computing growing while organizations need to respond to economic uncertainties. The Fourth Quarter 2022 Google Cloud Brand Pulse Survey emphasizes this as an important aspect to reach long-term business resilience. The aim of this collection is not to be exhaustive, rather to bring together many materials starting from the basics to helping with dealing with advanced topics.
Disclaimer: I work at Google in the cloud team. Opinions are my own and not the views of my current employer.
Update (#3) (10 June 2024): Several additional resources
Cloud FinOps is a framework that brings together people, process and technology promoting and driving financial awareness and accountability in the organization. It focuses on business value maximization rather than only cost optimization. The 5 minute video — What is FinOps and 3 reasons why you should care about it? — is a good introduction to the framework’s 5 building blocks making this possible:
- Accountability & enablement
- Measurement & realization
- Cost optimization
- Planning & forecasting
- Tools & accelerators
Read more about each of these building blocks in the article Cloud FinOps: The Secret To Unlocking The Economic Potential Of Public Cloud and learn more from cost-optimization experts’ recommendations and common industry techniques.
The Google Cloud FinOps landing page introduces the FinOps value, key considerations and how to measure its impact. As organizations grow their cloud usage they need to remain connected to the investments they are making and the value it is generating. Metrics can help with this and need to be defined across the 5 building blocks. This article proposes 5 key metrics to get started and measure Cloud FinOps impact in your organization.
But adopting FinOps successfully in an organization cannot be limited to processes and controls; it depends heavily on enabling the right cultural change across many different stakeholders in the organization.
Whitepapers
Getting started with FinOps on GCP
A framework for Google Cloud Platform (GCP), covering the teams required, the processes and behaviors to embrace, the outcomes expected, and an approach to implement it
The Business Value of Improved Performance and Efficiency with Google Cloud Platform (IDC)
IDC: How Google Cloud helps SMEs accelerate business growth through increase productivity and scalability while achieving greeter cost efficiencies
Maximize business value with cloud FinOps
Start from the core building blocks of cloud FinOps and define key success metrics for business value realization
This whitepaper aims to spotlight key insights that can be drawn from the data, highlighting how customers are generating measurable business value — also beyond cost savings — through their cloud transformation.
How to build financial resilience
Identify 3 foundational cloud pillars to help you accelerate time to value and become a more resilient company during an economic downturn: technology cost optimization, enhanced operational effectiveness & people-powered productivity and empowerment
Achieving cloud financial resilience with Cloud FinOpsAchieving cloud financial resilience with Cloud FinOps
From quick fixes to reduce your cloud spending to implementing effective cost governance, Google Clouds’ insights can help you develop your optimized, long-term cloud cost strategy.
Decoding Cloud FinOps to accelerate digital transformation
Digital transformation requires a programmatic approach through an incremental yet agile, cost-effective, value-driven, and sustainable strategy to drive successful transformation across the organization.
Getting started with FinOps on GCP
Implementing FinOps is not a one time activity but a continuous discipline and a journey that every organization should invest in as part of their cloud strategy. Getting started on the Cloud FinOps journey can be overwhelming and the following playbook may help to structure this for you. . Need help creating a concrete plan? A FinOps roadmap helps key stakeholders within a customer’s organization have the ability to answer the difficult question of “Where do we start?” Read more in this blog post.
Making information visible
Informing the teams with accurate data improves this understanding and facilitates moving to the next phase: making them accountable for managing and improving their cloud spend. A good starting point is the introduction of cost visibility dashboards. A couple of best practices to keep in mind include:
- Set up a billing export to BigQuery and create your data environment for easy analysis and visualization.
- Diving deeper into granular cost trends made users switch from the billing console to BigQuery requiring to write manual SQL queries to replicate the info that is diplayed. A new feature (released in May 2024) now provides an equivalent SQL query for every cost report in the billing console to simplify these cost drill downs.
- Augment billing data with labels and additional business related information for better reporting and advanced use cases in the future. Learn more about tagging and its benefits in the Driving Cloud FinOps at scale with Google Cloud Tagging whitepaper.
- Automate landing zone services as much as possible, not only project setup but also assigning labels. This will enable you to maintain consistency for your reporting processes.
My blog post Growing your Google Cloud Landing zone with data explains how data can enrich a landing zone and the role it can play in a FinOps framework. I make the case for a modular metadata platform to provide transparent, accurate and actionable data to all stakeholders within an organization, starting small and growing in phases. This concept is illustrated with an example that starts from Infrastructure-as-Code and transitions into FinOps.
- Getting started with GCP, the Google Cloud setup checklist helps you set up for scalable, production-ready, enterprise workloads. This checklist includes the tasks that you need to take into consideration to control and monitor your cloud environment. Next to security and access management, it also includes topics such as billing and resource hierarchy which play an important role in FinOps. The guide to Cloud Billing Resource Organization & Access Management will help you avoid common issues and enable best practices.
- Cost Management provides tools for monitoring, controlling, and optimizing your costs. This includes reports & dashboards, budget & alerts, recommendations and much more.
- Dashboard template and examples:
Billing usage and cost insights dashboard (live example)
Cost Optimization Dashboard (live example)
Visualize GCP Billing using Looker
Buy or build? A Framing Up FinOps case study on advanced reporting
Use Looker to empower Cloud financial decisions and FinOps (Next 23) - Identify configuration problems in Google Cloud with the Recommendations Hub
- Use Active Assist to optimize your cloud operations with recommendations to reduce costs, increase performance, improve security, and even help you make more sustainable decisions. Two Forrester reports on this page illustrate the cost savings and business benefits enabled by Active Assist. Sample recommendations include the VM Machine Type, Committed Use Discounts, Identifying idle VMs, Reduce overprovisioned Cloud SQL instances, BigQuery slot recommedations, etc.
- Unlock cloud savings with new Looker Studio Dashboard and Terraform scripts
The next part is then to implement a strong governance and operating model fostering a cost-conscious, empowered and accountable culture. Instead of seeing cost as someone else’s responsibility, it is accepted as their own responsibility. A crucial capability achieving this is call chargeback: the process of mapping cloud consumption to internal consumers within your organization and facilitating recovery of cloud services costs. This blog post explains how this can be implemented by combining detailed billing data exported from Cloud Billing with organizational data.
Whitepapers
Unlocking the value of cloud FinOps with a new operating model
Establish strong financial governance and create a cost-conscious culture by adopting a new operating model.
A guide to financial governance in the cloud
The importance of financial governance controls and the role these controls play in increasing the predictability of cloud costs
Cloud FinOps — Shared Services Cost Allocation
Cloud FinOps point of view on cost allocation of shared cloud services
Unit costing: The next frontier in Cloud FinOps
This paper examines the nature of and the need for cloud unit costing and examples.
Driving cloud efficiency
In Google Cloud, several options are available to perform workload as well as pricing optimizations. As with most things in technology, the greatest standards are only as good as how well they are followed. The limiting factor, more often than not, isn’t the capability of the technology, but the people and processes involved. This blog post identifies the principles for lasting success looking at effectively right-sizing deployments, using various tools at your disposal and a framework assisting you with prioritizing choices.
Whitepaper
Understand the principles of cloud cost optimization
Five ways to reduce overall cloud spend with a cost optimization strategy
As a best practice it is important to realize that cost optimization is not an objective of its own. Increasing spend accompanied by increasing business value will be positive; while increasing spend without business value raises concerns. Yet, even in positive cases, it remains important to find the best solution and associated cost to generate business value. Cloud offers many options to design and implement solutions, so are you making use of the most appropriate one? Is the selected architecture efficient? Have you optimized on the default settings? Are you not making use of options that are nice but not essential? But remember to also include the effort required for performing optimizations — it requires cost to reduce cost.
One of the 5 building blocks listed above is planning & forecasting. This is an important topic since it allows to track planned spend against actual spend, account for both steady state as well as new workloads and identify concerns when deviations occur. And generally, planning and tracking costs will be cheaper than fixing cost concerns.
- Cloud Financial Forecasting
- Demystifying Cloud Pricing
- Use the Google Cloud pricing calculator to estimate the workload cost
- Google Cloud console cost estimator
- Using the cost estimation API
To control cloud spend tools such as budgets, alerts and quotas can be used. A budget enables you to control the total cost by workload while the alert helps to keep track of the actuals. In this way, you can use budgets to automate cost controls:
- Protect your Google Cloud spending with budgets
- Stay informed by customizing your budgets
- Costs meet code with programmatic budget notifications
- Build better budgets using folders and organizations
- Keep a closer eye on Google Cloud costs with new Budgets for project users
- Cloud FinOps Powered by Generative AI
Quotas provide protection against cost overrun, and can be indicators of bad code. They protect the community of Google Cloud users by preventing unforeseen spikes in usage and overloaded services. But, they also help you manage resources. For example, you can set your own limits on service usage while developing and testing your applications to avoid unexpected bills from using expensive resources. Now, if you are looking for an automated way to manage quotas over a large number of projects, you can have a look at the Quota Monitoring Solution. This provides an easy and centralized way to view and monitor the quota usage in a central dashboard and to use default alerting capabilities across all quotas.
Each product has its own cost management best practices for which you can find more information in the documentation, in blog posts or videos. If you want to learn more about designing a cost-optimized foundation, you can start on the Cost optimization on Google Cloud for developers and operators page and explore the following:
- Best practices for optimizing your cloud costs
- Google Cloud Architecture Framework: Cost optimization
Diving more into detail, you can find an overview hereunder of several useful resources.
General
- Unlocking cloud cost optimization: A guide to Google FinOps Resources
- Framing up FinOps: How to optimize your cloud costs on Google Cloud
- Top 10 ways to lower your costs on Google Cloud
- Managing Cloud Cost Anomalies
Cloud billing
- Understanding billing
- Framing up FinOps: All about Google Cloud billing tools
- Understanding discounts
- Committed use discounts and sustained use discounts
- Flexible committed use discounts (documentation)
- Optimizing your cloud costs is even easier with Committed Use Discount recommendations
- Using Google Cloud’s new Pricing API
- Cloud Billing release notes (Learn about new features)
Whitepaper
Maximize Cost Efficiency with CUD Optimization Strategy
By incorporating best practices and committing to use key sets of Google Cloud services, organizations can save up to 70% on certain cloud costs.
Compute resource optimizations
- Optimizing your costs on Compute Engine (Youtube)
- 5 best practices for Compute Engine Cost Optimization
- Cost optimization best practices on Google Compute Engine (Youtube)
- Rightsize VM Recs (Bin packing)
- Idle VM Recs
- Idle Persistent Disk
- VM Scheduler (GKE autoscaler)
- Newer gen instances (custom machines)
- Spot VMs
- Modernize your business for less with AMD and Google Cloud (Next 24
- Optimize costs and efficiency with new compute operations solutions (Next 24)
GKE resource optimizations
- Sharing the inaugural State of Kubernetes Cost Optimization report
- Setting resource requests: the key to Kubernetes cost optimization
- How GKE cost optimization insights help you optimize Kubernetes
- Best practices for running cost-optimized Kubernetes applications on GKE
- GKE best practices
- GKE Cost Optimization
- Optimization in GKE
- GKE workload rightsizing — from recommendations to action
- Struggling to fix Kubernetes over-provisioning? GKE has you covered
- GKE Autopilot
- Comparing GKE Autopilot cost efficiency to traditional managed Kubernetes
- Introducing GKE cost estimator
- How to optimize Kubernetes for reliability and cost-efficiency (Next 23)
- Improve Kubernetes cost and reliability with the new Policy Controller policy bundle
- How not to sacrifice user experience in pursuit of Kubernetes cost optimization
- Elite performance in demand-based downscaling: The power of workload autoscaling
- Introducing granular cost insights for GKE, using Cloud Monitoring and Billing data in BigQuery
- Cloud FinOps and GKE: How Generali optimized spend with Google Cloud Consulting
Storage resource optimizations
- Best practices for Cloud Storage cost optimization
- Best practices for Cloud Storage
- Cost optimization best practices for Google Cloud Storage (Youtube)
- Object Lifecycle Management
- Object versioning
- Snapshot retention and cleanup
- Storage Classes (documentation)
BigQuery resource optimizations
- Cost optimization best practices for BigQuery — Google Cloud Community (online)
- Cost optimization best practices for BigQuery (blog)
- BigQuery Partitioning & Clustering
- Understand and optimize your BigQuery analytics queries
- Federation: Avoid duplication of data
- Data retention and clean up for active storage
- BigQuery Caching
- Flex Slots
- BigQuery Slots Recs
- Improve query performance and optimize costs in BigQuery using the anti-pattern recognition tool
- Introducing partitioning and clustering recommendations for optimizing BigQuery usage
- An insider’s guide to BigQuery cost optimization
- How to control spending with budgets and custom quotas
- Optimize your BigQuery data platform for performance and cost (Next 23)
Generative AI
Other resource optimizations
- Autoscaling
- The business case for managed databases in Google Cloud (Next 24)
- Cloud SQL Insights
The Business Value of Cloud SQL: Google Cloud’s Relational Database Service for MySQL, PostgreSQL, and SQL Server (IDC) - Unused IPs
- Optimize licensing cost
BYOL
Extended Memory - Networking cost optimization best practices
Network Service Tiers
Network Intelligence Center - Managing cost and reliability in fully managed applications
- A journey from App Engine to Cloud Run
- Optimizing cloud data lake cost with FinOps
- Cloud cost optimization for data pipelines
Predicting the cost of a Dataflow job
Dataflow insights
Introducing predictable cost options for Cloud Data Loss Prevention - Machine Learning
Best practices for performance and cost optimization for machine learning - BigTable Autoscaler
- Price out your SAP landscape on Google Cloud
Control cost and build financial resilience with automation: An SAP story (Next 23) - Accelerate cost savings with API management (Next 23)
- Cloud switching just got easier: Removing data transfer fees when moving off Google Cloud
Embedding FinOps and driving self-sufficiency
Google Cloud provides a number of training and certification programs that can help organizations get the most out of FinOps:
- Cost management in the cloud (Youtube video)
- Understanding Your Google Cloud Costs (Skills Boost)
- Optimizing Your Google Cloud Costs (Skills Boost)
- Cost Management with Google Cloud (Youtube video collection)
- Cost optimization podcast (Podcast)
It is important to highlight that this does not replace overall training and certification. From the start it is crucial to invest in skill development and continuous learning. This helps people to realize how decisions and actions influence costs and what actions can be taken to increase efficiency and effectiveness.
Creating internal communities to share experiences and best practices facilitates and enhances these learning experiences. Getting involved in external communities and learning from other organizations provides the additional benefits of having an inside-out view and may provide insights and inspiration for new initiatives:
- FinOps Foundation (community advancing people who practice the discipline of cloud financial management through best practices, education and standards) (Youtube channel)
State of FinOps - FinOps Sketchnotes:
1 — Introducing Cloud FinOps
2 — Meet the FinOps team
3 — Cost and value optimization on Google Cloud - Cloud FinOps and Cost Optimization Community
- When they go closed, we go open — Google Cloud and open billing data
FinOps Open Cost and Usage Specification (FOCUS), which aims to standardize cloud billing data into one common data schema
Standardize your cloud billing data with the new FOCUS BigQuery view - Google Cloud Next 2023 FinOps product announcements recap
- Google Cloud Next 2024 FinOps recap
- Google Cloud — How to optimize cloud spend with new FinOps tools
- Cloud FinOps and Cost Optimization Community
- Creating a FinOps culture change through cloud automation (Next 23)
- Unlock cloud value for everyone with Google Cloud FinOps tools (Next 23)
- Efficiency Engineering with Cloud FinOps at Target (podcast)
- Target Case: Measuring the Cost of an Application on GCP (Youtube)
- Google Cloud Next ’20 — OpenX case study (Youtube)
- Realizing cloud value for a render platform at Wayfair — Part 1
- Realizing cloud value for a render platform at Wayfair — Part 2
- FinOps from the field: Leveraging Cloud FinOps to measure the business value realized by the cloud transformation
- Optimize your Google Cloud costs with proven data-driven approaches (Next 23)
- Optimize your cloud with AI-based recommendations from Active Assist (Next 23)
- How to Optimize IT Costs (Datadog @ Next 24))
- Google Cloud Increases Speed of Cost Data and Takes a Curated Approach to Anomaly Management (Next 24) (Youtube)
- How MLB create granular cost insight and application insight with observability analytics (Next 24) (Youtube)
- Top five ways AI is making cloud FinOps practices easier to scale
- FinOps in Fashion: Why L’Oreal uses a business-first culture to build collaboration
- Runway for Startups and Scaleups: Optimizing Costs with Google Cloud Platform (presentation — capabilities)
- Ask Me Anything: How to Optimize Cloud Costs (Youtube)
- Cloud Cost Optimization Best Practices With Google Cloud (podcast)
- Cloud Billing best practices: How to manage your Cloud Billing Account and control your cloud costs (online / Youtube)
- Cloud Financial Management: How to foster a FinOps culture (online / Youtube)
- A Guide to Gamification for FinOps
And last but not least, stay up to date with the Cost Management blog.
Closing thoughts
It is important to realize that implementing with Cloud FinOps is not a one-time activity and that there is also not one single path to take. At every point along the way, you will need to reflect, fine-tune your approach and make decisions on how to continue your journey. Therefore, it is important to aim for progression rather than perfection: start small and continuously improve, but it is important to establish the necessary foundations early. The following themes can help you to define your strategy and decisions:
- Establish financial governance: Provide better cloud cost governance and controls to prevent cloud sprawl
- Provide cost transparency: Enable cloud cost visibility through persona-based reporting dashboards
- Improve forecasting accuracy: Shift from trend-based forecasting to driver-based forecasting models
- Optimize cloud spend: Embed continuous cost optimization with automation capabilities
- Accelerate business value: Align business objectives and digital strategies to drive cloud ROI
- Drive cost-conscious culture: Focus on enabling financial accountability across the organization by building a Cloud FinOps function