Google just released the Cloud Function 2nd Gen a couple of days ago. I am excited to see what it has to offer comparing the 1st Gen. There are a long list of changes in Google official page. In this topic, we only look at the major ones. Architecture In 1st…

Google Cloud Run is an excellence platform to run your application serverless. It comes with auto scaling feature at very low cost. The free-tier is pretty generous too. The best thing about Cloud Run is that it is a containerized platform. …

I have been working with different Reporting tools to get data from many different sources. One of the task that always come up now and then is to create a Custom Connector especially for REST API services. In this topic, we will do some quick comparison between the two popular…

Google Data Studio is a great tool to visualize your data. It is not only easy to use, it is free too. It supports all the major RDBMS including GCP BigQuery and Amazon Redshift. However, for other data sources such as REST API or OData, it does not have any…

In GCP PubSub, we could retain the acknowledged messages in the Subscription by setting the flag on. They could be kept up to 7 days or up to the ‘Message Retention Duration’, whichever is shorter. There are many reasons we would want to keep these. One of the most popular…

Google provides a few ways to load data to GCP BigQuery tables programmatically. One of the popular method is to use BigQuery API ‘insertAll’. However, if you like working with pandas DataFrame, you can directly load data from your DataFrame. First we take a look at querying into DataFrame. It…

Google Firestore is a NoSQL document database in GCP. If you are using Datastore, it is more or less the same. Google will auto-upgrade any exiting Datastore to Firestore. For new customer, Firestore will be the default choice. I am not sure what will happen if customer refuses to upgrade…

Bastion-host is the popular way to control access to your internal network. The way it works is very simple, every Bastion-host would be exposed to the Internet. We will connect to the Bastion-host and from there we can connect to any internal server (e.g. only has internal IP addresses). GCP…

Not every cloud resources are provisioned from a Terraform configuration. In production environment, most of the resources are actually outside of a configuration. For certain scenarios, we would want to move some existing resources into a Terraform configuration. The most common reason is that we want to provision some resources…