Sending Alerts to MS Teams from GCP Monitoring

In today’s cloud-centric world, efficient monitoring and rapid incident response are paramount. Google Cloud Platform (GCP) provides robust monitoring tools, but effectively communicating those alerts to your team can be a challenge. This blog post dives into the technical details of integrating GCP alerts with Microsoft Teams, enabling real-time notifications and streamlined collaboration within your team’s communication hub.

Why Integrate GCP Alerts with Microsoft Teams?

• Centralized Communication: Microsoft Teams likely serves as your team’s primary communication platform. Integrating GCP alerts directly into Teams channels consolidates critical notifications, eliminating the need to switch between platforms.
• Real-Time Notifications: Teams provides instant delivery of alerts, ensuring your team is aware of issues the moment they occur. This enables swift action and minimizes potential downtime.
• Enhanced Collaboration: Teams’ collaborative features allow your team to discuss the alert, share insights, and coordinate a response directly within the context of the notification.
• Customizable Notifications: You can tailor the format and content of alert messages to suit your team’s needs, highlighting the most critical information.

High-Level Architecture

Fig. Sending alerts from GCP monitoring to MS teams

• GCP Monitoring: GCP’s monitoring services (Cloud Monitoring, Cloud Logging, etc.) detect anomalies or issues within your cloud environment and trigger alerts.
• Pub/Sub: GCP’s Pub/Sub acts as the messaging backbone, reliably delivering alerts from various sources to a central topic.
• Cloud Functions: A serverless Cloud Function, written in your preferred language (e.g., Python, Node.js), processes alerts from the Pub/Sub topic.
  This function:
  i) Parses the alert payload, extracting key details.
  ii) Filters alerts (optional) based on your criteria.
  iii) Enriches alerts with additional information (optional).
  iv) Formats alerts into Teams-friendly messages.
• Microsoft Teams Webhook: The Cloud Function sends the formatted alerts to Microsoft Teams using an Incoming Webhook connector, displaying them as rich cards within a specified channel.

Technical Implementation

1. Microsoft Teams: Setting up the Incoming Webhook

• Refer to Microsoft teams documentation for setting up webhook.
• Navigate to the Channel: Open Microsoft Teams and locate the specific channel where you want to receive GCP alert notifications.
• Access Connectors: In the channel, click the “…” (more options) menu in the upper-right corner. Select “Connectors” from the dropdown list.
• Find and Add Webhook: In the search bar that appears, type “Incoming Webhook.” The Incoming Webhook connector should show up as an option. Click “Add” to add it to your channel.
• Configure Webhook: You’ll be asked to give your webhook a name. This can be something like “GCP Alerts” for easy identification. You can also optionally upload an image for the webhook.
• Generate Webhook URL: Click the “Create” button. Microsoft Teams will generate a unique URL specifically for this webhook. This is the URL you’ll use to send alert messages from your GCP Cloud Function. Copy this URL and store it securely.

2. GCP Cloud Functions: Building the Alert Handler

• Create a New Function: In the Google Cloud Console, navigate to the Cloud Functions service. Click on the “Create Function” button to initiate the process of creating a new function.
• Function Configuration:

Name: Provide a descriptive name for your function that clearly identifies its purpose, such as “gcp_alert_to_teams.”

Memory: Allocate the appropriate amount of memory based on the expected volume of alerts. (128MB is a good starting point for most cases).

Trigger: Select “Cloud Pub/Sub” from the available trigger types. This ensures that your function is automatically executed whenever a message arrives on the specified Pub/Sub topic.

Pub/Sub Topic: Choose the Pub/Sub topic that will be created in the next step (step 3). This is where GCP will send its alerts.

Runtime: Choose your preferred programming language. While the initial steps referred to Python, you can select any of the supported runtimes (Node.js, Go, etc.) based on your familiarity and preference.

Environment Variables: Create an environment variable named webhookURL. Paste the Microsoft Teams webhook URL you copied in step 1 as the value of this variable. This allows your function to access the URL securely without hardcoding it in your code.

• Code Implementation: Write the logic for your Cloud Function. Refer to the code in the git repo. This will include:

Parsing: Extract relevant data (e.g., incident ID, severity, resource name) from the JSON payload of the GCP alert message.

Filtering (Optional): If you want to receive only certain types of alerts in Teams, add logic to filter out unwanted alerts.

Enrichment (Optional): If needed, you can enhance the alert information by fetching additional data from other GCP services using API calls.

Formatting: Structure the alert data into a format suitable for Microsoft Teams cards.

• Sending: Make an HTTP POST request to the webhook URL stored in the webhookURL environment variable, sending the formatted alert data as the payload.
• Deploy the Function: Click on the “Deploy” button to deploy your Cloud Function to GCP. Ensure you grant the function the following permissions:
• Pub/Sub Subscriber: This permission allows the function to subscribe to the Pub/Sub topic and receive messages.
• Cloud Monitoring Viewer (or relevant service roles): This permission is necessary if your function needs to access additional information from other GCP services to enrich the alert data.

3. GCP Pub/Sub: Configuring Alert Routing

• Create a Topic: Go to the Pub/Sub section of the GCP console and create a new topic. This will be the designated topic where Cloud Monitoring sends its alerts.
• Create a Subscription: Still in the Pub/Sub section, create a new subscription to the topic you just created. Configure this subscription to trigger your Cloud Function whenever a new message is published to the topic.
• Configure Cloud Monitoring: Go to the Cloud Monitoring service and select the specific alert policy you want to route to Microsoft Teams. Edit the notification channels for this policy and add the Pub/Sub topic as a new notification channel.

Access and Permission

1. Google Cloud Platform (GCP):

• Cloud Monitoring Administrator:

Sets up and configures monitoring for GCP resources (VMs, databases, etc.).

Defines alert policies, thresholds, and notification channels.

Chooses the Pub/Sub topic for routing alerts.

• Pub/Sub Administrator:

Creates the Pub/Sub topic to which Cloud Monitoring will publish alerts.

Creates the subscription that links the topic to the Cloud Function.

Manages permissions for the topic and subscription.

• Cloud Functions Developer:

Designs and implements the Cloud Function code.

Handles alert parsing, filtering (if needed), enrichment (if needed), and formatting.

Deploys the Cloud Function and ensures it has the correct runtime environment and dependencies.

Tests the function thoroughly before and after deployment.

• Identity and Access Management (IAM) Administrator:

Creates a service account for the Cloud Function.

Grants the service account the necessary roles and permissions:

Pub/Sub Subscriber: To pull messages from the topic.

Cloud Monitoring Viewer: To view alert details (if enrichment is used).

(Optional) Other Service Viewer roles: If the function interacts with other GCP services for enrichment.

Manages and audits access to the service account.

2. Microsoft Teams:

• Teams Administrator:

Manages the Teams workspace and channels.

Creates the Incoming Webhook connector in the designated channel.

Ensures the webhook URL is securely stored and shared with the Cloud Functions developer.

Message Formatting for Microsoft Teams

The heart of the integration lies in crafting a Teams-friendly message that conveys essential information clearly and concisely. This is achieved by constructing a JSON payload adhering to the Microsoft Teams card message schema.

Key elements of a Teams card message:

• @type: "MessageCard" indicates this is a Teams card.
• @context: "http://schema.org/extensions" specifies the schema.
• themeColor: A color code (e.g., "FF0000" for red) to visually highlight the alert.
• title: A concise title describing the alert (e.g., "GCP Alert: High CPU Usage").
• summary: A brief overview of the issue.
• sections: This array contains one or more sections with:
• facts: Key-value pairs presenting critical alert details (e.g., "Severity: Critical," "Resource Name: instance-1").
• potentialAction: Optional buttons for actions like acknowledging the alert or linking to relevant documentation.

Conclusion

Integrating GCP alerts with Microsoft Teams is a game-changer for organizations relying on GCP for their infrastructure. It centralizes critical notifications, accelerates incident response, and fosters seamless collaboration, ultimately enhancing the reliability and performance of your cloud environment.

By understanding the technical nuances and following the outlined steps, you can create a powerful and efficient integration that empowers your team to proactively manage your cloud infrastructure.

Hope this blog helps! You can reach out to me on LinkedIn.