Stackdriver Monitoring Automation Part 4: Stackdriver Groups with Terraform

Published in

Google Cloud - Community

4 min readMar 5, 2019

In my quest for improving life through automation, I wrote a 3-part series about Stackdriver Monitoring Automation that used Google Cloud Deployment Manager to deploy Monitoring resources. I recently sat down to translate that automation into Terraform since Stackdriver Monitoring support has been added to Terraform. You can use these steps to automate the deployment of Stackdriver Monitoring resources in your environment using Terraform.

In this addition to the previous series, I have included the Terraform configuration for the same 3 scenarios described in the original Stackdriver Monitoring Automation series and won’t repeat the scenarios here. This post covers Stackdriver Groups while part 5 and part 6 cover Alerting Policies and Uptime Checks, respectively.

What’s available for automation in Terraform for Stackdriver Monitoring?

The following components are available as Terraform data sources and therefore can be used with the automation. The Monitoring coverage has expanded to cover the 4 main Stackdriver configurations also available via Monitoring API.

Stackdriver Groups
Alerting Policies
Notification Channels
Uptime Checks

Setting up Terraform for Monitoring

I decided to make use of Cloud Shell to run my Terraform scripts because Cloud Shell provides an easy command line interface and automatically provides the Terraform provider.

The first step is to configure the Google provider replacing each entry with the appropriate values for my GCP environment. Here’s the format described in the Terraform docs.

provider "google" {
  credentials = "${file("account_key.json")}"
  project     = "abab-cdcd-023991"
  region      = "us-central1"
  zone        = "us-central1-c"}

I removed the credentials line in my configuration because Cloud Shell provides the credentials by default and thus, didn’t require a service account key file. The IAM roles necessary for the deployment were already granted to me as the project owner. You will need the appropriate IAM permissions if you are not the project owner.

provider "google" {
  project = "abab-cdcd-023991"
  region  = "us-central1"
  zone    = "us-central1-c"
}

The Configuration Files

I created 3 separate groups: Apache, prod and qa based on the Stackdriver resource tags. All apache instances are included in the Apache Group based on the app=website tags. Only the instances tagged with env=qa and env=prod were included in the qa and prod Groups, respectively.

The qa and prod Groups specify Apache as the parent which tells Stackdriver Monitoring that these are subgroups. I used a reference to refer to parent names for each of the subgroups.

provider "google" {
  project = "abab-cdcd-023991"
  region  = "us-central1"
  zone    = "us-central1-c"
}resource "google_monitoring_group" "apache_parent" {
  display_name = "Apache"
  filter = "metadata.user_labels.app=has_substring(\"website\")"
}resource "google_monitoring_group" "apache_prod_subgroup" {
  display_name = "prod"
  filter = "metadata.user_labels.env=\"prod\""
  parent_name =  "${google_monitoring_group.apache_parent.name}"
}resource "google_monitoring_group" "apache_qa_subgroup" {
  display_name = "qa"
  filter = "metadata.user_labels.env=\"qa\""
  parent_name =  "${google_monitoring_group.apache_parent.name}"
}

You can find the main.tf files on the github repo.

charlesbaer/stackdriver-automation

The supporting github repo for the medium post on Stackdriver Monitoring automation. …

github.com

Creating the Groups and Subgroups

Adding the Stackdriver Groups requires 2 simple steps. First, initialize Terraform.

$ terraform initInitializing provider plugins...The following providers do not have any version constraints in configuration,
so the latest version was installed.To prevent automatic upgrades to new major versions that may contain breaking
changes, it is recommended to add version = "..." constraints to the
corresponding provider blocks in configuration, with the constraint strings
suggested below.* provider.google: version = "~> 2.1"
Terraform has been successfully initialized!

Then, the standard Terraform apply does all the work.

$ terraform apply
google_monitoring_group.apache_parent: Refreshing state... (ID: projects/abab-cdcd-023991/groups/4800045303134799755)
google_monitoring_group.apache_prod_subgroup: Refreshing state... (ID: projects/abab-cdcd-023991/groups/1346673955451905122)
google_monitoring_group.apache_qa_subgroup: Refreshing state... (ID: projects/abab-cdcd-023991/groups/2305611457471642078)An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  + createTerraform will perform the following actions:+ google_monitoring_group.apache_parent
      id:           <computed>
      display_name: "Apache"
      filter:       "metadata.user_labels.app=has_substring(\"website\")"
      name:         <computed>
      project:      <computed>+ google_monitoring_group.apache_prod_subgroup
      id:           <computed>
      display_name: "prod"
      filter:       "metadata.user_labels.env=\"prod\""
      name:         <computed>
      parent_name:  "${google_monitoring_group.apache_parent.name}"
      project:      <computed>+ google_monitoring_group.apache_qa_subgroup
      id:           <computed>
      display_name: "qa"
      filter:       "metadata.user_labels.env=\"qa\""
      name:         <computed>
      parent_name:  "${google_monitoring_group.apache_parent.name}"
      project:      <computed>Plan: 3 to add, 0 to change, 0 to destroy.Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.Enter a value: yesgoogle_monitoring_group.apache_parent: Creating...
  display_name: "" => "Apache"
  filter:       "" => "metadata.user_labels.app=has_substring(\"website\")"
  name:         "" => "<computed>"
  project:      "" => "<computed>"
google_monitoring_group.apache_parent: Creation complete after 1s (ID: projects/abab-cdcd-023991/groups/3905630164280032312)
google_monitoring_group.apache_qa_subgroup: Creating...
  display_name: "" => "qa"
  filter:       "" => "metadata.user_labels.env=\"qa\""
  name:         "" => "<computed>"
  parent_name:  "" => "projects/abab-cdcd-023991/groups/3905630164280032312"
  project:      "" => "<computed>"
google_monitoring_group.apache_prod_subgroup: Creating...
  display_name: "" => "prod"
  filter:       "" => "metadata.user_labels.env=\"prod\""
  name:         "" => "<computed>"
  parent_name:  "" => "projects/abab-cdcd-023991/groups/3905630164280032312"
  project:      "" => "<computed>"
google_monitoring_group.apache_qa_subgroup: Creation complete after 1s (ID: projects/abab-cdcd-023991/groups/7315859232068601075)
google_monitoring_group.apache_prod_subgroup: Creation complete after 2s (ID: projects/abab-cdcd-023991/groups/457136244756364624)

Once the Groups were created, I used the Stackdriver Monitoring console to verify that the Apache, qa and prod Groups had been successfully created. Notice that the prod and qa subgroups appear under the Apache group.

This concludes part 4 of the series. Read more about Stackdriver Monitoring Automation with Terraform in the other posts in the series and references below.

References: