MongoDB Data Change

How to listen for changes to a MongoDB collection.

So you’re doing something real-time?

Introducing MongoDB Change Streams

source: https://github.com/meteor/meteor-feature-requests/issues/158

Change Streams - MongoDB Manual 3.6

New in MongoDB 3.6
https://docs.mongodb.com/manual/release-notes/3.6/
Released January 10, 2018

$ mongod --version
db version v3.6.2

Change Streams are used in Database Mirroring

Database Mirroring: the replication of redundant copies of a database. The purpose is to ensure continuous data availability and minimize or avoid downtime.

Replication: a feature allowing multiple database servers to share the same data, thereby ensuring redundancy and facilitating load balancing. See Replication.

How it works:

1. replication creates additional copies of the data and allows for automatic fail-over to another node. Replication can also help with horizontal scaling.

This architecture in Mongo is also called a Replication Set.

You can then also make a replica set part of a sharded cluster.
— That’s for another post.

Replication Sets

Replica Set: A cluster of MongoDB servers that implements master-slave replication and automated failover.
More: https://docs.mongodb.com/manual/replication/

• For this example there will only be a master.

To use changeStreams the Mongo Database must be a Replication Set.

By default your Database will be a “Standalone” database.

Standalone: An instance of mongod that is running as a single server and not as part of a replica set.

In order to use changeStreams you must convert your “Standalone” database into a “Replica Set” database.

How to Convert a Standalone to a Replica Set.

1. Shut down the Standalone mongod instance.

kill -2 <MONGOD_PROCESS_ID>

2. Restart the instance as a Replica Set.
Use the --replSet option to specify the name of the new replica set.
The following command starts a standalone instance as a member of a new replica set named rs0.

mongod --replSet rs0

All options:

mongod --port 27017 --dbpath /srv/mongodb/db0 --replSet rs0 --bind_ip localhost,<ip address of the mongod host>

3. Connect to the mongodb with a mongo shell.

mongo

4. Initiate the new replica set.

rs.initiate()

About rs.initiate() command: https://docs.mongodb.com/manual/reference/method/rs.initiate/#rs.initiate

You should get the following, you can now exit mongo.

 rs0:PRIMARY>

Your database is now a member of a Replica Set!

You can check the status of the Replica Set with rs.status()

How To Use

The following example is a practical application for how you might use this.

LightBase

The following snippet is taken from the LightBase project;
a full working server Framework using everything posted here and more, check it out here.

Example:

When the Mongo Collection User changes in any way, all users are resent over the Socket.io connection.

https://gist.github.com/riodw/f1cdf875ee46e093e92ad917140eff95

Using: Node.js, MongoDB, Mongoose.js, Express.js, Socket.io

How It Works

First is to put a watch on the Mongoose Model

const changeStream = User.watch();

We can then use the event “Change” to get notified anytime the Collection changes.

changeStream.on(‘change’, function(change) {
    console.log(‘User COLLECTION CHANGED’);
    // CODE FROM BELOW GOES HERE
});

Now every time the database changes we can query for all the data and resend it.

    User.find({}, (err, data) => {
        if (err) throw err;
        if (data) { 
            // RESEND ALL USERS
            socket.emit(‘users’, data);
        }
    });

NOTE: you must start the mongodb with the --replSet rs0 everytime you start the database, otherwise it will default to a standalone.

./END

Useful links:
https://docs.mongodb.com/manual/tutorial/convert-standalone-to-replica-set
https://docs.mongodb.com/manual/tutorial/change-streams-example
https://docs.mongodb.com/v3.6/tutorial/change-streams-example
http://plusnconsulting.com/post/MongoDB-Change-Streams