How to migrate MongoDB Replica Set Cluster with zero downtime

Published in

BiTaksi Product & Tech

2 min readMar 18, 2022

Firstly, I want to say Merhaba to everyone. I’m working at BiTaksi as DevOps Engineer. BiTaksi is a hailing company in Turkey. MongoDB is one of the NoSQL database solutions at BiTaksi. I am going to show you how to migrate MongoDB Replica Set Cluster to new nodes in this article. Don’t worry :) I think you don’t need the installation guide for MongoDB since you already know. So, let’s check drawings about this process.

Phase-1

Create new nodes that can reach old nodes over the network.
Add new nodes to the replica set as hidden.

Phase-2

You can use the commands below for checking the data syncing process

Change RS configurations for the new nodes

Phase-3

Update the config of the application(s) that depends on MongoDB Cluster.

Old Connection URI:

mongodb://admin:super_strong_password@poc-mongo-01.bitaksi.com:27017,poc-mongo-02.bitaksi.com:27017,poc-mongo-03.bitaksi.com:27017/poc?replicaSet=poc&readPreference=secondaryPreferred&authSource=admin

Temp Connection URI:

mongodb://admin:super_strong_password@poc-mongo-01.bitaksi.com:27017,poc-mongo-new-01.bitaksi.com:27017,poc-mongo-new-02.bitaksi.com:27017,poc-mongo-new-03.bitaksi.com:27017/poc?replicaSet=poc&readPreference=secondaryPreferred&authSource=admin

Then you can remove old secondaries from the RS.

Phase-4

Change the primary node

Wait until the nodes are up-and-running

// you can check Replica Set status
rs.status()

Phase-5

Change your Connection URI in application configs again

Temp Connection URI

mongodb://admin:super_strong_password@poc-mongo-01.bitaksi.com:27017,poc-mongo-new-01.bitaksi.com:27017,poc-mongo-new-02.bitaksi.com:27017,poc-mongo-new-03.bitaksi.com:27017/poc?replicaSet=poc&readPreference=secondaryPreferred&authSource=admin

New Connection URI

mongodb://admin:super_strong_password@poc-mongo-new-01.bitaksi.com:27017,poc-mongo-new-02.bitaksi.com:27017,poc-mongo-new-03.bitaksi.com:27017/poc?replicaSet=poc&readPreference=secondaryPreferred&authSource=admin

And then, you can remove the old node from Replica Set

Conclusion

Why did we invent this process? It’s a good question. We had to migrate the cluster to a new network, so we need to trick for handling this migration without any outage.

Benefits

You have zero downtime. If you carefully follow these steps.
Sometimes background index can corrupt the nodes. If you have any issue with the index, you can get rid of it.

Drawbacks

You have to change application configs twice
Following those steps may become prone to error. So, make sure you take them carefully.
You have to do too many Replica Set configuration changes.

We didn’t find guidance like this. I wish it will help you. May the force be with you. 🚀