NodeJS 8 from Scratch — Part 6.1 — Api Development

In previous parts, we discussed about —

• basics of nodeJS
• debugging
• asynchronous programming
• call stack and event loop
• promises
• express and deployment
• testing with mocha and supertest

Part 1 — https://medium.com/@anujbaranwal/nodejs-8-from-scratch-part-1-a3c1431f1e15

Part 2 — https://medium.com/@anujbaranwal/nodejs-8-from-scratch-part-2-3035f8f46b09

Part 3 — https://medium.com/@anujbaranwal/nodejs-from-scratch-part-3-20956ec252a3

Part 4 — https://medium.com/@anujbaranwal/nodejs-from-scratch-part-4-d0aadf019c79

Part 5 — https://medium.com/@anujbaranwal/nodejs-8-from-scratch-part-5-3d3e3a84b64

Repo — https://github.com/anuj070894/nodejs_medium_2

In the next parts, we will discuss about developing apis for a Todo app in NodeJS with mongodb database including tests and also deploy it to heroku

So, let’s get started

We will do some setup initially and then we will discuss what all these terms mean

Installing MongoDB and Robomongo

• Grab the database from https://www.mongodb.com/
• Get the community server — it’s about >65MB in size. Extract it and keep it in a folder mongo and the bin folder inside it contains executables to connect to the database and start the mongoDB database, i.e., mongo/bin
• It is also required that we specify a data directory where the mongoDB can keep the database related files. So, let’s create a folder mongo-data adjacent to where we created mongo in the previous step
• Now, we are all ready to run the database server
• Do a cd mongo/bin and run ./mongod --dbpath ../../moongo-data , running this waits for any connection to be connected to the server. We can however run commands from the command line by running ./mongo/bin/mongo from another terminal. We can run sample queries from the terminal where we run mongo

By default, the database name is test which maps to the word — db in the statements below.

db.Todos.insert({text: "Sample Todo"});
db.Todos.find(); // every single item in the Todos collection

MongoDB is very forgiving because you can see here that we need not create a database. It creates once we make an insertion in the database.

Mongod is the daemon process for the mongodb system. It handles data request, manages data access, and performs background management operations. (A daemon process is a type of program on unix-like OS that is just running in the background, rather than under the direct control of user, waiting to be activated by the occurrence of a specific event or condition

Robomongo

A graphic user interface to manage your database. We can get it from https://robomongo.org/

We need to create a connection to local mongo database.

Before we go down any further, it is important to understand how the terms come together in the nosql vocabulary in comparison to the sql

NoSQL vocabulary

• test is the database here
• Todos is the collection(similar to Table in sql)
• {text: "Sample Todo"} is the document(similar to a record/row in sql)
• text is the field(similar to column in sql)

CRUD

We will be writing scripts to connect to the database and making CRUD(Create, Read, Update, Delete) operations to our database. To do this —

We will start with a node project

• So, run mkdir node-todo-api
• Run npm init

We will be using a library called mongodb to do these operations. It is the official mongoDB driver for nodeJS. Provides a high-level api on top of mongodb-core that is meant for end users. Later on, we will be using mongoose for our api that we will be deploying to production.

• Run npm install --save mongodb

in playground/mongo-connect.js

const {MongoClient} = require('mongodb'); // MongoClient lets you connect to a mongo server and lets you make request to the server

MongoClient.connect('mongodb://localhost:27017', (err, client) => {
if (err) {
return console.log('Error in connecting to the database ', err);
}
const db = client.db('TodoApp');
db.collection('Todos').insertOne({
text: "Sample Todo2"
}, (err, result) => {
if (err) {
return console.log('Error in inserting to the database ', err);
}
console.log(JSON.stringify(result, undefined, 2)); // {n: 1, ok: 1} .. n is the number of records being affected and ok: 1 means all went fine
});
client.close();
});

NOTE: We don’t need to create a database. If it doesn’t already exists, mongoDB creates a new database only when you first insert some data into it

After running this script with node playground/mongo-connect.js , we can see that in robomongo — a new document is inserted into the collection.

However, there is an extra _id property that is created by mongoDB. It is called as the ObjectId

The ObjectId property

• _id here is not an auto incrementing attribute, like we generally see in sql type databases. Mongo was designed to scale out easily. If it was an auto incrementing attribute, we would have to check at multiple places about the last value that was incremented
• _id is a random generated id which makes it easy to scale
• We can also get some details — timestamp of when it was created from _id

var {MongoClient, ObjectID} = require('MongoClient');
const id = new ObjectID();
console.log(id.getTimeStamp()); // will log the timestamp of when the constructor ObjectID was called

new ObjectID() — the 12 bytes value

• a 4 byte value representing the seconds since the Unix epoch
• a 3 byte machine identifier
• a 2 byte process id
• a 3 byte counter, starting with a random value

const {MongoClient, ObjectID} = require('mongodb'); // MongoClient lets you connect to a mongo server and lets you make request to the server
const id = new ObjectID();
console.log(id.getTimestamp());

Fetching Data

Just like we can insert into the database, we cal also fetch from the database.

in playground/mongo-find.js

Finding all the documents in the collection Todos

const {MongoClient, ObjectID} = require('mongodb'); // MongoClient lets you connect to a mongo server and lets you make request to the server
MongoClient.connect('mongodb://localhost:27017', (err, client) => {
if (err) {
return console.log('Error in connecting to the database ', err);
}
const db = client.db('TodoApp');
db.collection('Todos')
.find() // returns a cursor/pointer
.toArray((err, items) => {
if (err) {
return console.log('Error in finding from the database ', err);
}
console.log(JSON.stringify(items, undefined, 2)); // [{}, {}] .. collection of documents in an array
});
client.close();
});

Finding by query

.find({completed: true}) // returns a cursor/pointer

Finding by id

.find({_id: new ObjectID("5b54938ccc416d730d62a013")})

NOTE: We need to use ObjectID. With plain id as string, it won’t work.

Branch — 01_Init

Deleting documents

Now, we will be looking at queries that will help us deleting from the database

in playground/mongo-delete.js

To delete one document that matches the query first —

const {MongoClient, ObjectID} = require('mongodb'); // MongoClient lets you connect to a mongo server and lets you make request to the server
MongoClient.connect('mongodb://localhost:27017', (err, client) => {
if (err) {
return console.log('Error in connecting to the database ', err);
}
const db = client.db('TodoApp');
db.collection('Todos').deleteOne({
text: "Sample Todo2"
}, (err, result) => {
if (err) {
return console.log('Error in inserting to the database ', err);
}
console.log(JSON.stringify(result, undefined, 2));
});
client.close();
});

To delete all the documents that matches the query —

db.collection('Todos').deleteMany({
text: "Sample Todo2"
})

However, it can be seen by logging the results that none of them actually gives the document in the log. Sometimes, it is needed in the delete operation that we know what record is getting deleted.

db.collection('Todos')
.findOneAndDelete({
text: "Sample Todo2"
}, (err, result) => {
if (err) {
return console.log('Error in inserting to the database ', err);
}
console.log(JSON.stringify(result, undefined, 2));
});

Result —

{
    "lastErrorObject": {
        "n": 1
    },
    "value": {
    ... document
    },
    "ok": 1
}

Updating documents

Updating documents requires a bit more parameters. Let’s see how to do that.

in playground/mongo-update.js

findOneAndUpdate(filter, update, options, callback) —

• filter — document selection filer
• update — update operations to be performed on the document
• options — optional settings

const {MongoClient, ObjectID} = require('mongodb'); // MongoClient lets you connect to a mongo server and lets you make request to the server
MongoClient.connect('mongodb://localhost:27017', (err, client) => {
if (err) {
return console.log('Error in connecting to the database ', err);
}
const db = client.db('TodoApp');
db.collection('Todos').findOneAndUpdate({
text: "Sample Todo2"
}, {
$set: {
text: "Sample Todo3"
}
}, {
returnOriginal: false
}, (err, result) => {
if (err) {
return console.log('Error updating the record ', err);
}
console.log(JSON.stringify(result, undefined, 2));
});
client.close();
});

So, that’s how we do update with mongoDB.

Branch — 02_Init

That’s pretty much in this part 1 of Api development with MongoDB and NodeJS. In the next parts, we will learn how to use mongoose ORM — which is really great to give structure to your data.

See ya later. :)

Part 6.2 — https://medium.com/@anujbaranwal/nodejs-8-from-scratch-part-6-2-api-development-f92f76eb3521

Part 6.3 — https://medium.com/@anujbaranwal/nodejs-8-from-scratch-part-6-3-api-development-9b046fed7364

Part 6.4 — https://medium.com/@anujbaranwal/nodejs-8-from-scratch-part-6-4-api-development-38d600a35ad9