All You Need to Know About MongoDB on Google Cloud

Google Cloud offers a marketplace with many stacks & service solutions with Click to Deploy. The solutions includes database solutions(mongodb, cassandra, mysql) to Blog & CMS(wordpress, joomla) along with Developer tools(gitlab, jenkins). They are not a fully managed solution and google doesn’t offer support for any but the stacks decreases time to set up solutions and one can enjoy the services by paying only for the GCP resources but not for the solutions*.

Among the deployments offered on Google Cloud Launcher, mongodb is one which can be deployed at a click with creation of primary, secondary instances (compute engines) in a replica set along with arbiter node.

How to Create MongoDB Replica Set

Head over towards MongoDB Launcher on Google Cloud Console for deployment of the NoSQL database in cluster of instances. Click on “LAUNCH ON COMPUTE ENGINE”.

Configure deployment, replica set name and disk type on the console. Choose the number of nodes among which one will be primary replica and rest secondaries. Also, choose a arbiter node which can be a small instance as it is utilized just for choosing the primary replica by voting. Its recommended to choose SSD for the data storage which has high IOPS & throughput. Grab details about the performance difference from official optimization guide.

Now, within few minutes, the clusters are up not only with the number of replicas specified and arbiter but also with a separate disks for each node which has naming standard of mongodb-servers-vm-0-data that are attached to respective instance. The data on those disk are synced with each other. Also, by selecting External IP option None, we can hide the nodes from outside the specific subnet.

Connecting with MongoDB ReplicaSet

There are different ways of connecting to the mongodb replica set:

Connecting to each node: On the Compute Instances page on console, hit the SSH button and new window opens with ssh connection. Also, SSHing into google cloud instances is easy with gcloud sdk: gcloud compute ssh [Instance Name] --zone [zone] --project [project]

On arbiter node:

raju@mongodb-arbiters-vm-0:~$ mongo
MongoDB shell version v3.4.14connecting to: mongodb://127.0.0.1:27017MongoDB server version: 3.4.1

On primary node:

raju@mongodb-servers-vm-0:~$ mongo
MongoDB shell version v3.4.14 connecting to: mongodb://127.0.0.1:27017 MongoDB server version: 3.4.14 
.....
rs0:PRIMARY>

On secondary:

raju@mongodb-servers-vm-0:~$ mongo
MongoDB shell version v3.4.14
.....
rs0:SECONDARY>

Connect to ReplicaSet

As we have primary and secondary replicas running, to establish mongodb connection with the replica set rather than each one, enter this command on any of the node:

raju@mongodb-servers-vm-1:~$ mongo “mongodb://mongodb-servers-vm-0,mongodb-servers-vm-1/myDB?replicaSet=rs0”

This way, we get connected to the primary replica/node at first and in case of fail over, the arbiter node chooses another node as primary and the connection is established to new primary.

Fail Over Demo

First, we connect to the replica set from arbiter node

raju@mongodb-arbiters-vm-0:~$ mongo "mongodb://mongodb-servers-vm-0,mongodb-servers-vm-1/myDB?replicaSet=rs0"
MongoDB shell version v3.4.14 connecting to: mongodb://mongodb-servers-vm-0,mongodb-servers-vm-1/myDB?replicaSet=rs0
I NETWORK  [thread1] Starting new replica set monitor for rs0/mongodb-servers-vm-0:27017,mongodb-servers-vm-1:27017
I NETWORK  [thread1] Successfully connected to mongodb-servers-vm-1:27017 (1 connections now open to mongodb-servers-vm-1:27017 with a 5 second timeout)
I NETWORK  [ReplicaSetMonitor-TaskExecutor-0] Successfully connected to mongodb-servers-vm-0:27017 (1 connections now open to mongodb-servers-vm-0:27017 with a 5 second timeout)
rs0:PRIMARY>

Here, the instance mongodb-servers-vm-0 is primary and mongodb-servers-vm-1 is secondary. Now, lets restart mongod service on the primary node

raju@mongodb-servers-vm-0:~$ sudo service mongod restart

On the arbiter connection:

I NETWORK  [thread1] Successfully connected to mongodb-servers-vm-1:27017 (1 connections now open to mongodb-servers-vm-1:27017 with a 5 second timeout)
W NETWORK  [thread1] No primary detected for set rs0
...
W NETWORK  [thread1] No primary detected for set rs0
rs0:PRIMARY>

Now, we are connected to the instance mongodb-servers-vm-1 which has become the primary replica now. Check the replica set status:

raju@mongodb-arbiters-vm-0:~$ mongo
rs0:ARBITER> rs.status()

We get the status:

"members" : [ {
"_id" : 0, 
"name" : "mongodb-servers-vm-1:27017",                        "health" : 1,
"state" : 1, 
"stateStr" : "PRIMARY",
.....
"electionTime" : Timestamp(1525084250, 1),
},
{ "_id" : 1,
"name" : "mongodb-servers-vm-0:27017",                        "health" : 1,
"state" : 2,
"stateStr" : "SECONDARY",
.....
"syncingTo" : "mongodb-servers-vm-1:27017",
}

MongoDB ReplicaSet Connection from Compute Engine Instances

From any of the Google Cloud compute engine instances, the mongodb connection can established by passing the replicaset host name: mongodb://mongodb-servers-vm-0,mongodb-servers-vm-1/myDB?replicaSet=rs0

But we cannot get connected to the set by providing the ip address of any of the nodes. The name of the replicas can be found on /etc/hosts of respective instance.

MongoDB ReplicaSet Connection from Localhost

Sometime, we may need to connect to the replicaSet launched on Google Cloud from local computer, for that we need to create SSH tunnel:

gcloud compute ssh --ssh-flag=-L27017:localhost:27017 --project=PROJECT --zone=ZONE INSTANCE_NAME

By that way, we can connect to replicaSet even from MongoDB Compass for exploring and manipulating data.

Monitoring MongoDB Cluster

As they say Make life easier with Ops Manager, its one of the best solution provided by MongoDB which offers performance visibility to query optimization to backup and alerts out of the box. Ops Manager can be downloaded for various platforms but I faced issues with ubuntu and later succeeded for CentOS without any hurdle.