Set Up Google Cloud DataProc Job Monitoring

In this post, we’ll setup Google DataProc job monitoring so that we can get notified whenever a DataProc job fails.

Note: We’ll setup all resources in us-central1 region and our DataProc cluster is also supposed to run in us-central1 region.

Step1: Google Cloud doesn’t have a service similar to AWS SES, so we’ll be using a third party service from Google Cloud market place.

Go to Marketplace and subscribe to SendGrid Email API and generate API key. SendGrid’s free plan allows us to send 12000 mail per…




AVM Consulting — Clear strategy for your cloud

Recommended from Medium

Azure Cross Tenant Access (Authentication, Authorization, Private Communication)

READ/DOWNLOAD!? Foundations of Software Testing: ISTQB Certification FULL BOOK PDF & FULL AUDIOBOOK

Career In PowerApps

How can the right eForm solution grow your business?

Readable Test Code

Decision Control Structures in C

Using Python with braces


Linux Basic History

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Vinayak Pandey

Vinayak Pandey

Experienced Cloud Consultant with a knack of automation. Linkedin profile:

More from Medium

Create CI/CD pipeline with GitLab and Cloud Build

Tutorial: Bucket Replication for Google Cloud Platform (GCP) Cloud Storage

Continuous Delivery using Google Cloud Build — deploying a microservice to Google Kubernetes…

SFTP using CyberDuck via IAP Tunnel in Google Cloud Platform