Master GCP Monitoring with Datadog: Google Cloud integration with Datadog

Introduction

This article demonstrates the integration process of Google Cloud with Datadog, illustrating how to effectively monitor your Google Cloud environment using Datadog.

We will be using Datadog free trial account for this demo.

Lets Get started

Go to https://www.datadoghq.com/ and Click on Free Trial.

2. Fill Required details and sign up

Now Create Datadog Principal

1. In Datadog, navigate to the Integrations > Google Cloud Platform .
2. Click on Add GCP Account. If you have no configured projects, you are automatically redirected to this page.
3. If you have not generated a Datadog principal for your org, click the Generate Principal button.

Copy your Datadog principal and keep it for the next section

Now , In order to integrate Datadog with Google Cloud, it’s essential for Datadog to access Logging and Monitoring data from Google Cloud. To establish this connection, setting up authentication between these platforms is crucial, which involves utilizing a Service Account.

Create your Google Cloud service account

1. Open your Google Cloud console .
2. Navigate to IAM & Admin > Service Accounts.
3. Click on Create service account at the top.
4. Give the service account a unique name, then click Create and continue.
5. Add the following roles to the service account:

• Monitoring Viewer
• Compute Viewer
• Cloud Asset Viewer
• Browser

Click Continue, then Done to complete creating the service account.

1. In Google Cloud console , under the Service Acounts menu, find the service account you created in the first section .
2. Go to the Permissions tab and click on Grant Access
3. Paste your Datadog principal into the New principals text box.
4. Assign the role of Service Account Token Creator and click Save.

Now Validate Infrastructure List ,Showing Availavle VM Instance

Serverless Tab to See Metrics related to Cloud Run and Cloud Function

Now we will see how to send Google Cloud logs to Data Dog

1. Go to the Cloud Pub Sub console and create a new topic.
2. Give that topic an explicit name such as export-logs-to-datadog and Save.
3. Go back to the Pub/Sub that was previously created, and add a new subscription.
4. Select the Push method and enter the following https://gcp-intake.logs.us5.datadoghq.com/api/v2/logs?dd-api-key=<YOUR_DATADOGA_API_KEY>&dd-protocol=gcp
5. Verify if above datadog URL is correct for your datadog instance and replace with your datadog URL.
6. Hit Create at the bottom.
7. Go to Log Router and filter the logs that need to be exported.
8. Hit Create Export and name the sink accordingly.
9. Choose Cloud Pub/Sub as the destination and select the Pub/Sub that was created for that purpose. Note that the Pub/Sub can be located in a different project.
10. Hit Create and wait for the confirmation message to show up.
11. Lets wait for sometime to receive logs.

Now we can analysze all logs in Datadog.

Refer below video for demo

About Me

As an experienced Fully certified (11x certified) Google Cloud Architect with over 7+ years of expertise in Google Cloud Networking,Data ,Devops, Security and ML, I am passionate about technology and innovation. Being a Champion Innovator and Google Cloud Architect, I am always exploring new ways to leverage cloud technologies to deliver innovative solutions that make a difference.

If you have any queries or would like to get in touch, you can reach me at my email address vishal.bulbule@techtrapture.com or connect with me on LinkedIn at https://www.linkedin.com/in/vishal-bulbule/. For a more personal connection, you can also find me on Instagram at https://www.instagram.com/vishal_bulbule/?hl=en.

Additionally, please check out my YouTube Channel at https://www.youtube.com/@techtrapture for tutorials and demos on Google Cloud.