Get object notification for Google cloud storage using pub/sub

Whenever we store data in Google cloud storage, many times we aim to get updates for every change in bucket. May it be retrieving recently added file, size,timestamps etc. For this Cloud Pub/Sub Notifications allow you to track changes to your Cloud Storage objects.

Subscribing to Cloud Pub/Sub Notifications is a versatile way to trigger alerts and actions in response to changes in a bucket. Now let’s see how we roll!

First of all, let’s forward fast the prerequisites for this task:

1. Enable the cloud pub/sub API
2. Have enough bucket permissions (In case you don’t own project, use ACLs or IAM & have owner permission or storage.buckets.update permission resp.)
3. Have sufficient permission to access pub/sub topics

Diving to main scene! Hoping you have all ingredients require to prepare the dish..

1. Let’s create a bucket:

gsutil mb gs://[UNIQUE_BUCKET_NAME]

2. Create a Cloud Pub/Sub topic and bind it to our GCS bucket:

gsutil notification create -t [TOPIC_NAME] -f json -e OBJECT_FINALIZE gs://[BUCKET_NAME]

-t specifies pub/sub topic

-f specifies that you want the payload of the messages to be get as object in JSON API.

You can also specify your other event types by using -e

3. Now let’s create a Pub/Sub subscription:

gcloud beta pubsub subscriptions create [SUBSCRIPTION_NAME] --topic=[TOPIC_NAME]

Great! We have all things ready jack! Just need to read those messages. Here is sample which I am using for my task.

Are you ready to taste dish? There you go! Run the script as:

python [SCRIPT_NAME].py [PROJECT_ID] [SUBSCRIPTION_NAME]

And try some adventures on bucket, you should get output like following:

What to do with these events notifications?

Now that’s a clever question! So you can retrieve this pub/sub message and add to your application,get notified via e-mail and so on. Have a look at this example which gives fair idea about using pub/sub notifications with app engine.

In the next write, I will share how to accomplish same task using cloud functions. Till then have a happy life!