Centralised Monitoring in GCP

Monitoring in Google Cloud with Metric Scopes

Why Monitoring ?

Monitoring is an integral part of any application or service, whether it be on-prem or cloud, every resource or component must have the metrics defined for it to be monitored in order to ensure that everything functions as expected.

With respect to Google Cloud, Cloud Monitoring which is one part of Google Cloud’s Operation suite, is integrated with most Google Cloud services, and it automatically collects and stores performance information about those services. It can also collect system and application metrics from third-party applications.

Google Cloud Monitoring

By default, a Google Cloud project has visibility only to the metrics it stores. However, you can expand the set of metrics that a project can access by adding other Google Cloud projects to the project’s metrics scope. The metrics scope defines the set of Google Cloud projects whose metrics the current Google Cloud project can access.

Approach of Centralised Monitoring

Administrators prefer to view the dashboard, metrics associated with resources on a single place to avoid any back and forth actions on the Google Cloud Console. It lets to view and manage the metrics in a single project, multiple projects within an organization, multiple projects across multiple organizations and multiple Google cloud projects and AWS Accounts.

Photo by Miguel A Amutio on Unsplash

Scoping Project and Monitored Project

Scoping project is the one which hosts the metric scopes. The projects for which resources has to be monitored is called monitored projects. As a best practice the scoping project should be isolated and shouldn’t have any resources. Administrators can add projects as monitored project within the scoping projects. The scoping project stores information about its metrics scope. It also stores the alerts, uptime checks, dashboards, and monitoring groups that you configure for the metrics scope. You can identify the scoping project for a metrics scope as the project selected by the Google Cloud console project picker.

Setup Centralised Monitoring

1. Ensure that your Identity and Access Management (IAM) roles on the scoping project and on each project that you want to add as a monitored project include all permissions in the Monitoring Admin (roles/monitoring.admin) role.
2. Open the Google Cloud Console and from the project picker choose the Project which you would want to make it as Scoping Project.
3. In the Monitoring navigation pane select settings icon and in GCP Projects page click “Add GCP Projects”.
4. In the Add Google Cloud projects page, click Select Projects, and then make your sections. To save your changes, click Select. You are returned to the GCP Projects page, and the table on that page is updated to list your selections.
5. After you add projects to a metrics scope, it takes about 60 seconds for changes to propagate through all Monitoring systems. Before you create a chart or alerting policy, wait at least 60 seconds.

Using Terraform :

resource "google_monitoring_monitored_project" "primary" {
  metrics_scope = "my-project-name"
  name          = google_project.basic.project_id
}

resource "google_project" "basic" {
  project_id = "m-id"
  name       = "m-id-display"
  org_id     = "123456789"
}

name : Resource name of the Monitored Project.

metrics_scope: The resource name of the existing Metrics Scope that will monitor this project. Example: locations/global/metricsScopes/{scoping_project_id/number}

Limitations:

Cloud Monitoring officially supports up to 375 Google Cloud projects per metrics scope.

You can add up to 1,000 Google Cloud projects per metrics scope , but you might experience performance issues, especially when querying custom metrics or historical data. Cloud Monitoring guarantees performant queries and charts only for 375 Google Cloud projects per metrics scope.

Happy Learning!