In this story we will assume that you have designed and implemented a Node-RED solution that you wish to put into production hosted on Google Cloud Platform (GCP). Here we will look at some practices that we can employ to make that possible in a secure manner.

When we deploy a Node-RED solution, we will actually be deploying two parts. We will be configuring the Node-RED runtime and also the solution we want to host on that run-time. The technique we will be illustrating here is the creation of a custom Docker image that will package together:

  • Node-RED.
  • Any custom…

One of the features of Dialogflow CX is its ability to record its operations to GCP Cloud Logging. What this means is that when a request is submitted to Dialogflow for intent matching, a log record is written. When Dialogflow responds, a second log record is generated. Each of the instances of a log record is a structured entity containing a rich set of fields described through JSON. Included in the data are a wide variety of items including:

  • The date/time that the log was written.
  • The Dialogflow session id used to relate multiple interactions as being part of the…

When we set up a VPC network, we may not wish to allow Internet connectivity from that network. What this can mean is that applications running attached to that network may not be able to reach Google managed services such as Cloud Storage, Pubsub, BigQuery and many others. These services are exposed with public (Internet) IP addresses. Further, if there are on-premises applications that wish to avail themselves of the Google managed services, we may also wish to route on-premises traffic to our GCP network through VPN or other interconnect solutions and again we appear to have the same issue.

Google’s Document AI service allows you to process documents and parse out their content into structured and machine readable data. When we think about documents here, think about scanned documents as opposed to Google or Word documents which already have structured content. Instead, think about documents that may have hand-written or typed values entered into them. Examples of documents in this class may include:

  • W2 earnings statements
  • 1099 income declarations
  • Your driver’s license or passport
  • The current medications form you fill in when visiting a new doctor

The Document AI product/service is described in detail here. The service is intended…

A public IP address is one that is addressable from the Internet. Within a GCP environment, we can declare that Compute Engines should never be assigned public IP addresses. This can be defined at the Organization level and becomes an organization policy. Policies are policed across all the Projects belonging to that Organization including Compute Engines that may be created by other GCP products as part of their own execution. The reason why we may wish to disable public IP addresses is that this will reduce the attack surface. If a Compute Engine in GCP has no need to be…

A GCP user wanted to create a Memorystore Redis attached to a VPC. This was easy enough. The user then wanted to create another VPC in a separate project and have VMs in that VPC access the original Memorystore Redis. Can this be achieved and if so how?

This article walks us through a journey where we set up such an environment and got it working. The story takes us through the working to get there and explains some of the puzzles involved.

We will start with a simple diagram of our base peered network:

Google Cloud Platform has a product to provide managed Apache Airflow. That product is called Cloud Composer. In addition, GCP has a product to provide managed Apache Beam. That product is called Cloud DataFlow. As part of the GCP documentation there is a tutorial that can be followed that takes the reader through the process of setting up a Cloud Composer solution that will read a comma separated values text file and insert each of the rows contained within into a BigQuery table. The work is orchestrated by Composer with the insertion into the database performed by Dataflow.

The instructions…

When we use the GCP product called Cloud Functions, we are supplying the body of a function that contains the code logic we wish to have executed. By doing this, we are separating ourselves from any concern or implementation of how that function is invoked. It is Cloud Functions that causes the execution of our code when incoming requests arrive. We do not have to develop any form of serving scaffolding. Cloud Functions also takes care of starting up as many instances as we need based on load and scaling down to zero when no calls are in-flight.

An alternative…

Google Cloud Storage (GCS) provides blob storage for data. Files can be uploaded to GCS and subsequently retrieved. The storage is cheap and provides excellent availability and durability. GCS provides a variety of programming language APIs that can be used by custom applications and many of Google’s products are pre-built to produce and consume data to and from GCS. Command line tools such as gsutil also provide scripting access. Data can automatically be ingested if it is web addressable using the Storage Transfer product.

What is not provided by Google in the out of the box GCS story is the…

Geocoding is the notion of taking an address and determining information about its location on the Earth. This is typically the geo position given by latitude and longitude coordinates. Recently, a CDAP/Data Fusion user had a list of addresses as input and needed to enrich the data with geocoding information. While Google Maps provides an easy to use API to perform this task, there was nothing baked into CDAP that would allow us to leverage this API. …

Neil Kolban

IT specialist with 30+ years industry experience. I am also a Google Customer Engineer assisting users to get the most out of Google Cloud Platform.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store