Google Cloud Associate Certified Engineer — 100daysoflearning (Part 9)
Day 75–79
In continuation to the course for Google Cloud Associate Certified engineer lets move on with next set of learning topics.
The next section that we are going to look is how to install google cloud SDk locally , create VCN and its related resources, create public and private buckets , create pubsub topic, create Bigtable and create Bigquery.
Though you can see the cloud sdk prebuilt in the cloud console itself and you can do most of the stuff from there only but what if you want you code , scripts etc to run form your machine to the cloud. So for that installing Google Cloud SDK on the machine is the first task and then configuring it to the specific project form you google cloud console.
Google Cloud SDK installation : Installing google cloud is really easy on any platform, be it Linux, Mac or windows. I will be demonstrating for Windows here.
I will follow he official documentation so it upto date .
Official link — https://cloud.google.com/sdk/install
For windows — https://cloud.google.com/sdk/docs/quickstart-windows
After downloading the installer its the regular process of doing next and install the google cloud sdk.
After opening in the google cloud shell run ‘gcloud init’
When you do gcloud init with a new configuration you get a URL and it automatically opens up in browser as well in order to autheticate the google cloud shell with your account .
When you login to the account in the browser it will ask for the required permissions :
Once you click allow your google cloud shell is authenticated with your google cloud account and you will be able to start using it .
Also once you are authenticated you will see in the cloud shell that is running on your machine , with list of projects and create option :
you can choose existing projects or create a new one. For now I have selected an existing project. I can start using the gcloud commands and running scripts from my local machine in order to play around with my Google cloud account with the project that I have chosen.
Note -The end goal here is to deploy an application with frontend, backend, Google cloud services etc. In the process of doing that the first thing is to setup the Google Cloud shell and connecting it to the Google Cloud account(which is done).
The application uses a lot of Google cloud API’s and services and I will install then one by one . In the end we will see how the application works. But for this particular article the focus is on enabling api’s and creating Network using the google cloud SDK via bash scripting.
Lets first clone the repository that have all the code base for the scripts and the application .
git clone https://github.com/saiyam1814/la-ace-find-seller.git
Enabling the API’s with script which in-turn are using the gcloud commands:
As you can see above gcloud service enable <service id> command is use to enable the api in the cloud from the Google cloud shell that is running or your machine. You can also loop over them to make it more geeky. In order to check what all services are enabled you can type the command :
gcloud services list --enabled --sort-by=”NAME”
Once you run the enableapis.sh you will see the below output meaning the api’s are enabled:
VCN creation :Next we will create a network in Google Cloud using the gcloud commands.
As you can see in the file first is the network create command which will be a custom network.
Next two commands are for creating two different subnets with different CIDR Range (10.29.0.0/24 and 10.28.0.0/24).
Next is for allowing all the traffic for the subnets and last one for ssh traffic. Though for a production environment you would not do this and only allow restricted amount of traffic from certain ports to certain ip ranges so that the security is maintained.
You will notice there are few variables used in this file like $PRODUCT_SUBNET etc these variables are present in project_settings.sh. I will fill out variables for the current project and unique bucket values and then run network.sh .
As you can see the all the commands have executed successfully and printed out the status of created as well. Lets see in the google cloud console.
The new VPC network has been created with subnets and firewall rules. Though everything can be done via command line but when it comes to automation this is where the google cloud sdk is powerful . Suppose you want to spin up entire infrastructure, you can easily write scripts and run them without going via the UI.
Bucket Creation:After creating the VPC next lets go ahead and create a bucket. the command used to create the bucket is as follows :
gsutil mb -p $PROJECT_NAME -c regional -l $PROJECT_REGION gs://$PRIVATE_ASSETS/
Here the gs commands means we have something to do with google storage , mb means make bucket , regional for setting the bucket region and then the name of the bucket (variables are coming from the projectsetup.sh file used earlier). Also one more thing to note here is that google buckets are private by default so no need to specify private argument. Lets go ahead and run this file.
Now I will also create a public bucket for that you need to run the following:
As you can see above there are additional steps for creating a private bucket. one is setting up the CORS so that the bucket url do not get inaccessible due to CORS issue and chrome/browser treats them as valid url’s. Next is just running of a create bucket command which will create a private bucket. After the create the gsutil cors command is set on the bucket and at last the user access is changed in order to make that bucket publicly accessible to the internet. Lets run this file and see the public bucket getting created.
As you can see the above bucket has been created.
Pubsub Creation: Next on the list is to create pubsub so that we can publish messages to it. creating a pubsub a very easy command , we will just create a pubsub topic and its then ready to be used to send messages.
after running the file it will create the pubsub and it can be verified by the command : gcloud pubsub topics list which result in below output:
Bigtable creation: Next on the list is creation of big table now this script if you see is big and you do not need to understand every piece of it, the import things in this file are the gcloud command to create the instance , ‘cbt’ commands to create the table and region of creation as not all services are there in all regions.
once you run the bigtabe.sh you can see the bigtable instance list in ready state.
Bigquery creation: Last component creation that I will show you in this article is Big query creation . Its a pretty simple script :
First is to create a big query component using the bq , setting the location to US then the make(mk) for creating the dataset
Second is to create a table definition that was setup by bigtable script. Lets run this and create bigquery.
So we have enabled Google API’s, setup a new VCN , Subnet , set its firewall rules , allowed ssh connection as well, created public and private buckets, created a pubsub, created bigtable, created bigquery . This all has been done via command line ie the Google Cloud SDK and its very powerful and much more developer friendly as far as automation is concerned or for running files located on system onto the Google cloud console. Now we have installed all the common components required for the application , in the next article we will start installing the application and its related resources.
Happy Learning.
Saiyam Pathak
https://www.linkedin.com/in/saiyam-pathak-97685a64/
https://twitter.com/SaiyamPathak
