A Complete MongoDB PWA Tutorial

JavaScript Teacher
Apr 5, 2020 · 13 min read
Image for post
Image for post

I always liked tutorials that show you practical examples. They help understand how to use code to actually build apps in production environment. Then I could take that knowledge to build my own. In this tutorial we will store world geolocation data in a Mongo collection.

Open the demo of this tutorial at infinitesunset.app live photography app.

I open-sourced this entire project on GitHub

Check complete source code for this Express.js SSL MongoDB server. This is a good starting point for someone who’s trying to get Mongo to work on remote host. ⭐Star it, 🍴 fork it, ⤵️download it or ✂️copy and paste it.

What Is Mongo?

The name Mongo was taken from the word Humongous.

In context of storage it can mean dealing with large quantities of data. And that’s what databases are pretty much for. They provide you with an interface to create, read, edit(update) and delete large sets of data (such as user data, posts, comments, geo location coordinates and so on) directly from your Node.js program.

By “complete” I don’t mean documentation for each Mongo function. Rather, an instance of Mongo code used as a real-world example when building a PWA.

A Mongo collection is similar to what a table is in a MySQL database. Except in Mongo everything is stored in a JSON-like object (But also uses BSON to store unique _id property similar to how MySQL stores unique row key.)

Installing Mongo And Basic Operations

In Terminal on Mac or bash.exe on Windows 10, log in to your remote host with ssh root@xx.xxx.xxx.xx (replace the x’s with your web host IP address.) Then use cd to navigate to your project directory on remote host (/dev/app) for example.

Install the system-wide Mongo service:

sudo apt-get install -y mongodb

The -y directive will automatically answer “yes” to all installation question.

We need to start the mongo service after this (explained in next section.)

Mongo Shell.

Just like MySQL command line shell where we create tables, users, etc. Mongo has its own shell to create collections, show databases and so on.

First we need to start Mongo service:

sudo service mongodb start

If you ever need to stop mongo server you can do that with:

sudo service mongodb stop

(but don’t do that, we need the service running for next part!)

To enter the Mongo shell type the mongo command into your cli/bash:

mongo

You will see your bash change to > character. Let’s type use command to create a new (or switch to existing) a new database:

> use users
switched to db users

Type > db to verify that we indeed switched to users database:

> db
users

Let’s take a look at all existing databases by executing show dbs command:

> show dbs
admin 0.000GB
local 0.000GB

Even though we created users database in earlier step, it’s not on this list. Why? That’s because we haven’t added any collections to the database. This means that users actually exists but it won’t be shown on this list until a collection is added.

Adding Mongo Collection

Let’s add a new collection to users database. Remember a collection in mongo is the equivalent of a table in MySQL:

db.users.insert({name:"felix"})

We just inserted a collection into users database.

Let’s run > show dbs now:

> show dbs
admin 0.000GB
local 0.000GB
users 0.000GB

Simply inserting a JSON-like object into users database has created a “table.” But in Mongo it’s just part of the document-based model. You can insert more objects into this object and treat them as you would rows/columns in MySQL.

Installing Mongo NPM Package

npm install mongodb --save

Generating The Data

Before we go over Mongo methods let’s decide what we want to store in the database Let’s take a look at world coordinate system. It’s quite different from Cartesian. The central point here is located relatively near Lagos, Nigeria:

Image for post
Image for post
Latitude and Longitude visualized. [0,0] point is located in the center of the coordinate system with axis going left and down in negative direction. HTML5 geo location will take care of calculating your location automatically based on your IP address. But we still need to convert it to 2D on the image. Image credit.

Front-End

Collecting the data.

HTML5 provides out-of-the-box geo location and gives us latitude and longitude as long as the client agrees to share that information. The following code will create a pop up message box on desktop and mobile devices asking user to “allow” to share location.

If user agrees, it will be stored in lat and lon variables:

if (navigator.geolocation) {
navigator.geolocation.getCurrentPosition(function(position) {
let lat = position.coords.latitude;
let lon = position.coords.longitude;
}
}

Well, that’s a great start.

But this isn’t enough. We need to convert it to 2D coordinate system HTML5 <canvas> understands. We need to draw location markers on top of the map image! So I wrote this simple World2Image function that takes care of that:

function World2Image(pointLat, pointLon) {
const mapWidth = 920;
const mapHeight = 468;
const x = ((mapWidth / 360.0) * (180 + pointLon));
const y = ((mapHeight / 180.0) * (90 - pointLat));
return [x, y];
}

We simply divide map image dimensions by 360 and 180 respectively and then multiply them by 180 + pointLong and 90 — pointLat to adjust to center. And our final code that converts latitude/longitude to 2D coordinates will look like:

if (navigator.geolocation) {
navigator.geolocation.getCurrentPosition(function(position) {
let lat = position.coords.latitude;
let lon = position.coords.longitude;
let xy = World2Image(lat, lon);
let x = xy[0];
let y = xy[1];
}
}

In your app of course you can use any data you want. We just need a meaningful set to demonstrate a practical example for a potential live sunset photography app.

Place the code above between <script> tags within <head> tag in your PWA. Preferably inside: window.onload = event => { /* here */ };

Now every time a new visitor joins our page, they will be asked to share geolocation. Once they press “Allow” button, it will be collected and stored in lat, lon, x and y vars. We can then create a Fetch API call to send it to our back-end server.

API End-point Round Trip

Below are sections: Front-End, Back-End and Front-End again. Developing an API end-point (/api/user/get for example) you will often follow this pattern. First we need to call Fetch to trigger an HTTP request to an end-point.

Front-End API

Here is the simple code that gives us two Fetch API requests. At this time we only need these two actions to build our geolocation API. Later in this article I’ll show you how they connect to Mongo on the back-end.

class User {
constructor() {
/* Currently unused */
this.array = [];
}
}

// Make JSON payload
let make = function(payload) {
return { method: 'post',
headers: { 'Accept': 'application/json',
'Content-Type': 'application/json' },
body: JSON.stringify(payload) };
}

/* /api/add/user
Send user's geolocation to mongo
payload = { x : 0, y : 0, lat: 0.0, lon: 0.0 } */

User.add = function(payload) {
fetch("/api/add/user", make(payload))
.then(promise => promise.json()).then(json => {
if (json.success)
console.log(`Location data was entered.`);
else
console.warn(`Location could not be entered!`);
});
}

/* /api/get/users
Get geolocations of all users who shared it */

User.get = function(payload) {
fetch("/api/get/users", make(payload))
.then(promise => promise.json())
.then(json => {
if (json.success)
console.log(`Location data was successfully fetched.`);
else
console.warn(`Users could not be fetched!`);
});
}

We can now use static User functions to make Fetch API calls. Of course, nothing is stopping you from simply calling fetch function anywhere in your code where it makes sense. It’s just putting everything into User object helps us organize code.

Back-End

This tutorial assumes you have Node and Express running. Showing how to set that up would take a whole another tutorial. But here are the core API commands coded in Express. It’s quite simple.

The code below is from express.js file — the complete SSL server. The only thing you need to change if you plan on implementing this on your remote host is generating your own SSL certificates with certbot and LetsEncrypt.

Express, Multer and Mongo Init Shenanigans

First we need to initialize everything:

/* include commonplace server packages... */
const express = require('express');
const https = require('https');
const fs = require('fs');
const path = require('path');

/* multer is a module for uploading images */
const multer = require('multer');

/* sharp works together with multer to resize images */
const sharp = require('sharp');

/* instantiate the express app... */
const app = express();
const port = 443;

// Create Mongo client
const MongoClient = require('mongodb').MongoClient;

// Multer middleware setup
const storage = multer.diskStorage({
destination: function (req, file, cb) {cb(null, './sunsets')},
filename: function (req, file, cb) {cb(null, file.fieldname + '-' + Date.now())}
});

// Multer will automatically create upload folder ('./sunsets')
const upload = multer( { storage : storage } );

// body-parser is middleware included with express installation. The following 3 lines are required to send POST body; if you don't include them, your POST body will be empty.
const bodyParser = require('body-parser');
app.use(bodyParser.urlencoded({extended: true}));
app.use(bodyParser.json());

// Create some static directories. A static folder on your site doesn't actually exist. It's just an alias to another folder. But you *can also map it to the same name. Here we're simply exposing some folders to the front end.
app.use('/static', express.static('images'));
app.use('/sunsets', express.static('sunsets'));
app.use('/api', express.static('api'));
app.use('/static', express.static('css'));
app.get('/', function(req, res) { res.sendFile(path.join(__dirname + '/index.html')); });

// General use respond function -- send json object back to the browser in response to a request
function respond( response, content ) {
const jsontype = "{ 'Content-Type': 'application/json' }";
response.writeHead(200, jsontype);
response.end(content, 'utf-8');
}

// Utility function...convert buffer to JSON object
function json( chunks ) { return JSON.parse( Buffer.concat( chunks ).toString() ) }

We still need to add end points to express.js file. The code below is assumed to go into the same file just underneath what we’ve written above.

Setting up API end-points

Creating POST API end-points with Express is a breeze. The code below will be triggered on the server when we use fetch calls we talked about earlier (using the User object and its static methods) from client-side.

API End-point 1 /api/add/user

This server side end-point will add geolocation data to Mongo database.

// End-point: api/add/user
// POST method route - Add geolocation to Mongo's Photographer collection

app.post('/api/add/user', function(req, res, next) {
const ip = req.connection.remoteAddress;
const { x, y, lat, lon } = req.body;

// Connect to mongo and insert this user if doesn't already exist
MongoClient.connect(`mongodb://localhost/`, function(err, db) {
const Photographer = db.collection('Photographer');

// Check if this user already exists in collection
Photographer.count({ip:ip}).then(count => {
if (count == 1) {
console.log(`User with ${ip} already exists in mongo.`);
db.close();
} else {
console.log(`User with ${ip} does not exist in mongo...inserting...`);
let user = { ip: ip, x: x, y: y, lat: lat, lon: lon };

// Insert geolocation data!
Photographer.insertOne(user, (erro, result) => {
if (erro)
throw erro;

// console.log("insertedCount = ", result.insertedCount);
// console.log("ops = ", result.ops);

db.close();
});
}
});
res.end('ok');
});
});

API End-point 2 /api/get/users

This server side end-point will grab a list in JSON format of all currently stored geolocations in the Mongo database.

// End-point: api/get/users
// POST method route - Get all geolocations from Mongo's Photographer collection

app.post('/api/get/users', function(req, res, next) {

// Connect to Mongo server
MongoClient.connect(`mongodb://localhost/`, function(err, db) {
const Photographer = db.collection('Photographer');
Photographer.find({}, {}, function(err, cursor) {

// NOTE: You must apply limit() to the cursor
// before retrieving any documents from the database.

cursor.limit(1000);
let users = [];

// console.log(cursor);
cursor.each(function(error, result) {
if (error) throw error;
if (result) {
// console.log(result);
let user = { x: result.x,
y: result.y,
lat: result.lat,
on: result.lon };
// console.log("user["+users.length+"]=", user);
users.push(user);
}
});

// A more proper implementation: (WIP)
(// End-point: api/get/users
// POST method route - Get all geolocations from Mongo's Photographer collection

app.post('/api/get/users', function(req, res, next) {

// Connect to Mongo server
MongoClient.connect(`mongodb://localhost/`, function(err, db) {
const Photographer = db.collection('Photographer');
Photographer.find({}, {}, function(err, cursor) {

// NOTE: You must apply limit() to the cursor
// before retrieving any documents from the database.

cursor.limit(1000);
let users = [];

// console.log(cursor);
cursor.each(function(error, result) {
if (error) throw error;
if (result) {
// console.log(result);
let user = { x: result.x, y: result.y, lat: result.lat, lon: result.lon };
// console.log("user["+users.length+"]=", user);
users.push(user);
}
});

// A more proper implementation: (WIP)
(async function() {
const cursor = db.collection("Photographer").find({});
while (await cursor.hasNext()) {
const doc = await cursor.next();
// process doc here
}
})();

setTimeout(time => {
const json = `{"success":true,"count":${users.length},"userList":${JSON.stringify(users)}}`;
respond(res, json);
}, 2000);
});
});
});function() {
const cursor = db.collection("Photographer").find({});
while (await cursor.hasNext()) {
const doc = await cursor.next();
// process doc here
}
})();

setTimeout(time => {
const json = `{"success":true,"count":${users.length},"userList":${JSON.stringify(users)}}`;
respond(res, json);
}, 2000);
});
});
});

API End-point 3

This will upload an image from an HTML form via Node multer package.

// End-point: api/sunset/upload
// POST method route - Upload image from a form using multer

app.post('/api/sunset/upload', upload.single('file'), function(req, res, next) {
if (req.file) {
// console.log("req.file.mimetype", req.file.mimetype);
// const { filename: image } = req.file.filename;

let ext = req.file.mimetype.split('/')[1];
let stamp = new Date().getTime();
// console.log("ext=",ext);
output = `./sunsets/sunset-${stamp}.${ext}`;
output2 = `https://www.infinitesunset.app/sunsets/sunset-${stamp}.${ext}`;
console.log("output=",output);
// console.log("output2=",output2);
sharp(req.file.path)
.resize(200).toFile(output, (err, info) => {
// console.log(err);
// console.log(info.format);

});
// fs.unlinkSync(req.file.path);
res.end(`<html><body style = 'font-size: 50px'><img src = "${output2}" style = "width: 100%;"><div style = "text-align: center; margin-top: 50px;">Picture uploaded! <a href = 'https://www.infinitesunset.app' target = 'sunset'>Go Back</a></div></body></html>`,`utf-8`);
}
// req.file is the `avatar` file
// req.body will hold the text fields, if there were any

res.end(`<html><body>Something went wrong.</body></html>`,`utf-8`);

MongoDB Command Overview

Primarily we will use .insertOne and .find Mongo collection methods. When inserting a location entry, we will test if that entry already exists in the database. If it does, there is nothing to do. If it doesn’t, we’ll insert a new entry.

You can also use Mongo’s .count method to count the number of entries in the document that match a particular document filter.

To do any Mongo operations we first need to connect to our Mongo server from within of our Node application! This is done using the .connect method.

.connect

MongoClient.connect(mongodb://localhost/, function(err, db) {
/* mongo code */
};

db.close()

When we’re done we need to close the database connection:

MongoClient.connect(mongodb://localhost/, function(err, db) {
db.close();
};

.insertOne

Photographer.insertOne(user, (erro, result) => {
if (error)
throw error;

// console.log("insertedCount = ", result.insertedCount);
// console.log("ops = ", result.ops);


Don't forget to close db
db.close();
});

Here results.ops will display the object that was inserted.

.find

The find method produces something called a cursor in Mongo. This cursor is an object that you can call .each on to iterate over all found items. Per Mongo documentation you should always set a limit first with cursor.limit(1000);

Photographer.find({}, {}, function(err, cursor) {

// NOTE: You must apply limit() to the cursor
// before retrieving any documents from the database.

cursor.limit(1000);

let users = [];

// console.log(cursor) is [Cursor object];
cursor.each(function(error, result) {

// check for errors
if (error)
throw error;

// get data
let x = result.x;
let y = result.y;

/* etc... */
}
}

Front-End

Displaying the data.

Now that we have basic Mongo API running on our back-end, we can send data back to the front-end and the client side needs to visually display it on the screen.

I’ve chosen to use HTML5 <canvas> to draw the coordinates as images. To draw images on canvas you first have to include them in your HTML document as regular images with IMG tag. To prepare images let’s include to cache them in the browser:

<style type = "text/css">/* Load images but move them offscreen */
#you, #world, #mark { position: absolute; top: -1000px; left: -1000px }</style>

<img src = "you.png" id = "you"/> <-- your location -->
<img src = "world.png" id = "world" /> <-- world map image -->
<img src = "marker.png" id = "marker" /> <-- location marker -->

And now the core of our front-end. This call will do a bunch of things: see if device browser supports HTML5 geolocation property, create a 2D canvas and draw world map image on it, get HTML5 geolocation data, send it to Mongo using Fetch API, make another call to get all previously inserted items into our mongo db on the back-end and animate canvas by drawing markers based on returned JSON object:

<script type = "module">

// Create payload -- redundant everywhere,
// because all POST payloads are JSON
// now in neat make() function

let make = function(payload) {
return { method: 'post',
headers: { 'Accept': 'application/json' },
body: JSON.stringify(payload) };
});

// safe entry point, all images loaded etc.
window.onload = function(event) {

// a place to store location markers
window.markers = [];

// Because this is <script type = "module",
// to make User object globally available so
// it can be used in HTML attribute events,
// we have to manually add it to window object

window.onload = U => {
window.User = User;
}

// create 2D canvas
const c = document.getElementById("canvas");
const ctx = c.getContext("2d");

// get our images
const world = document.getElementById("world");
const here = document.getElementById("here");
const marker = document.getElementById("marker");

if (navigator.geolocation) {
navigator.geolocation.getCurrentPosition(
function(position) {

let lat = position.coords.latitude;
let lon = position.coords.longitude;
let xy = World2Image(lat, lon);
let x = xy[0];
let y = xy[1];

console.log(`You are at ${lat} x ${lon} or = ${x} x ${y}`);

// Add this user to mongo
User.add({ x : x, y : y, lat: lat, lon: lon });

// get our user locations
fetch("/api/get/users", make({})).then(
promise => promise.json()).then(json => {
/* Store returned list of markers */
window.markers = json.userList;
// optionally debug each entry:
json.userList.forEach(photographer => {
// console.log(photographer);
});
// Remove the loading bar
window["loading"].style.display = 'none';
// display number of items returned
console.log("json.userList.length", json.userList.length);
});

// animation loop that draws all locations in real time
function loop() {
if (here && ctx) {
// draw the world map
ctx.drawImage(world, 0, 0, 920, 468);
ctx.save();
// move to the center of the canvas
ctx.translate(x, y);
// draw the image
ctx.drawImage(here, -here.width/2, -here.width/2);
// draw marker images
ctx.translate(-x, -y);
if (window.cameras) {
window.cameras.forEach(cam =>
ctx.drawImage(polaroid,
cam.x-polaroid.width/2,
cam.y-polaroid.width/2));
}
ctx.restore(); // restore context
}
requestAnimationFrame(loop);
}
requestAnimationFrame(loop);
});
} else msg.innerHTML = "Geolocation is not supported by this browser.";
}
</script>

Note…you don’t have to render your data in this exact way. If you are using React library just go ahead and use components. It doesn’t matter. Here I used vanilla JavaScript just for this example. What matters are the fetch requests and that we are rendering the returned JSON object containing location data on the screen. You don’t have to use canvas. You could have used just the DIV elements for each marker.

Open the demo of this tutorial at infinitesunset.app live photography app.

I open-sourced this entire project on GitHub

Check complete source code for this Express.js SSL MongoDB server. This is a good starting point for someone who’s trying to get Mongo to work on remote host. ⭐Star it, 🍴 fork it, ⤵️download it or ✂️copy and paste it.

End Result (also see it running live at www.InfiniteSunset.app )

Thanks for reading, I hope you found this Mongo tutorial insightful.

Image for post
Image for post

Check out my book JavaScript Grammar for common use JavaScript.

Or simply follow me on Twitter for coding tutorial announcements etc.

The Startup

Medium's largest active publication, followed by +756K people. Follow to join our community.

JavaScript Teacher

Written by

Issues. Every webdev has them. Published author of CSS Visual Dictionary https://amzn.to/2JMWQP3 few others…

The Startup

Medium's largest active publication, followed by +756K people. Follow to join our community.

JavaScript Teacher

Written by

Issues. Every webdev has them. Published author of CSS Visual Dictionary https://amzn.to/2JMWQP3 few others…

The Startup

Medium's largest active publication, followed by +756K people. Follow to join our community.

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store