From Zero to Hero — run parse-server on Google Cloud Platform Part 1— Run parse-server and MongoDB locally with docker-compose

This blog post is part 1 of a series of blogs on how to run parse-server on GCP container engine. Before you start reading this blog please refer to part 0 of the series which is the introduction part.

In this part i will show how you can run parse-server + MongoDB containers on your local machine via docker compose. If you already familiar with Docker or parse-server please feel free to skip the theoretical parts.

Docker and Docker Compose (Theoretical)

We will use docker and specifically we will use docker compose to run the app on our local machine. Docker is the most popular software container platform and allow developers to run their apps and solutions in a “production like environment”. Docker works with containers each container contain all the things that you need to run your software and unlike vm’s are not tight to full operating system.

The basic steps to containerized your app are:

1. Create a Dockerfile — Dockerfile contain dependencies, volumes, Working directories and commands that needs to be executed by docker tool in build time.
2. Docker build — the build command will access your Dockerfile and will start to run the commands in it line by line so the order of the commands inside the Dockerfile are important. The result will be a docker image that will contain all the code, settings and resources which are required to run your app
3. Docker run — the docker run command will check for the image that you want to run, create a container from that image and finally run it.

Dockerhub

With docker you can run multiple containers that can talk (linked) with each other (somebody said “microservices”?) and thanks to one of the biggest ecosystem in the software world, usually you don’t even need to build such images on your own, you can just download them (pull them) from dockerhub which is a registry (or repository) with hundreds of thousands of docker images ready to use. So, before you create an image on your own go and check there if this image is already exist.

Docker Compose

Docker Compose is a tool which allows you to build and run multiple containerized docker application in one command. In order to use this tool you first need to create some yml file which contains all the images that you want to run, their configurations and the relationships between them. In our case we will use docker-compose that will contain two images:

1. parse-server image that we will build on our own
2. MongoDB image which will be the persistence of our app. This image will be pulled from dockerhub.

The parse-server image will be linked to the MongoDB image. By linking those two image together we tell docker that the parse-server container needs to “talk” with the MongoDB container.

parse-server (Theoretical)

parse-server is an open source BaaS (backend as a service) solution that was originally developed by the parse.com team. The aim of parse.com was to let developers focusing on building the client app (either in iOS/Android/Web and more) and their platform will take care the server side (backend) part. parse.com became very fast the most popular platform for building such apps and that’s the reason why they were acquired by Facebook.
2 years after the acquisition Facebook decided that parse.com services are not part of their strategy and they decided to shut down the service. They notify to all of their hundreds of thousands of customers to migrate their data and either host it on their own or seek for another solution.

The best thing that came out from this story is the parse-server open source. parse.com team decided to open source their solution and name it parse-server. At the beginning they were very involved and provide very good support to customers by helping them to migrate their parse.com apps to parse-server but now they are not active anymore.

parse-server is 100% written in NodeJS and use MongoDB as the main Database. parse-server takes care to all the things related to backend development in order to let developers focusing on building the front-end apps. parse-server is very stable, well documented and is being maintained by some developers from the community (and i am one of them :) ) .
It will take me couple of hours to list all the features of parse-server so i will focus only on the most important ones:

1. Auto client class creation — This feature allows you to create parse objects directly from the client so developers don’t even need to go and create the relevant collection(s) in MongoDB. When you want to create a new object simply create it via one of the client SDK’s. This is also relevant for adding/removing properties to a collection and create relationships between them.
   BTW! some of the developers don’t really like this feature due to security issues so in parse-server you can easily shut down this feature if you want.
2. User management — parse-server provide very powerful user management system. With this feature you can: login users, register new users, login users and link them with 3-party provides like: facebook, google, github, instegram, twitter and more.
3. ACL’s — ACL stands for access control list. Parse use ACL’s to protect objects from being read/modified by users that are not allowed to.
4. Plugin base architecture — most of the features in parse-server are built via plugins so if you have some requirement which is not fulfilled by the current solution you can (maybe) create an adapter that will replace the default one. For example: let’s assume that you want to allow your users to login with snapchat you can easily create auth adapter , implement all the relevant code, provide some configurations and then add it in the parse-server configuration object and from now on your users will also be able to login with their snapchat account.
5. GeoPoints— using parse-server you can easily create geo fields that will store the location of a user/business in the world. Moreover, via GeoQuery you can execute advanced queries like: show me list of users which are up to 100 miles away from my current location and more.
6. Push notifications — with parse-server you can easily send push notifications to one or more users via the platform SDK’s. It works by creating an installation object for each one of the user devices so each installation stores the device token and some more details about the user device.
7. Live Query — Live Queries allowing you to subscribe to changes in the database in realtime . If you are familiar with other solutions like RethinkDB or Firebase real time database then you will easily understand this concept as well. In live query you can subscribe to changes in MongoDB and then parse-server notify you in realtime when this object has been changed. In my app i use this feature to know my friends presence (when they are online/offline). The live query is based on WebSocket and work very fast. If your app need to work in large scale you can integrate the parse live query server with a pub/sub solution. Currently only Redis pub/sub, RabbitMQ and GCP pub/sub are available so if you have some other favorite Pub/Sub solution you can simply create an adapter and use it.
8. Client SDK’s / Rest API — parse-server client SDK’s are available for the most popular programming languages: iOS, Android, JS/NodeJS, PHP, .NET. Unity and more. In part of the client SDK’s you can use local data store which provide you auto sync capabilities and allows you to create offline first applications. If you are not a fan of SDK’s parse-server REST API is available as well.
9. Cloud Code Functions and Hooks — Cloud Functions allows you to write your own custom logic and run them on parse-server. cloud functions are written in NodeJS and can be either simple or complex, the only thing that is important to understand is that cloud functions are stateless: you send a request, parse-server process this request and return a response.
   Hooks allows you to write some code that will be triggered by the platform in response to some generic events like: BeforeSave,AfterSave,BeforeDelete,AfterDelete so if for example you want to run some custom logic before a User is being saved in the DB simply create a BeforeSave hook on the user object.

Like i mentioned above, these are only part of the capabilities of parse-server. You can read more about other features like: email integration, logging, file storage support, media files and more in the official docs website

NodeJS and NPM (Theoretical)

The node package manager is a very powerful tool and allows NodeJS developers to reuse code by adding ready to use NodeJS modules to their project. There are millions of modules that you can add which can help you to accelerate your development and reduce your TCD (total cost development) in more than a half. Moreover, npm is also a dependency manager means that when you add an npm module to your project it will automatically add all the other modules that this module depends on.

MongoDB (Theoretical)

parse-server use MongoDB as the persistent layer so all the data that you create, update, delete and read will be persist into MongoDB. MongoDB is a very powerful and the most popular NoSQL database our there. MongoDB store data in binary document JSON objects (aka DocumentDB).
MongoDB is very fast, easy to scale, schemaless and easy to use. Like i mentioned above one of the greatest features of parse-serve is the ability to create/generate collections on the fly and it can be done because MongoDB allowing that.

Please note! MongoDB is a very powerful database but it’s not the best fit for any use case. Since MongoDB is NoSQL it doesn’t support transactions and in part of the applications transactions are mandatory because using transactions you make sure that your data is always consistent and you can run multiple queries which are depended on each other and wrap them in one transaction something that MongoDB cannot do.

Prerequisites Installations

Docker

The first thing that we need to install is Docker which will install the docker CLI and docker-compose tools on your machine. Luckily, docker team created an amazing installers for Mac and Windows that you can be installed in a click of a button.

In order to install docker on your local machine go to the download page on the docker store, Click on the Get Docker CE for Mac (Stable) button and follow the instructions on the docker store site. This installer will install the docker toolbox and the docker CLI on your local machine.

After installing docker you will see the popular docker whale on your mac status bar.

taken from docker store

Another way to check that everything works is to open your mac terminal and run the docker command and if you see the following:

it means that docker is installed on your machine.

Node.js + node package manager (npm)

The best place to download npm + NodeJS is from Node.js official website.Go to the Node.js download page, click on the v7.10.0 current green button to download the Node.js installer and execute the installer to install Node.js on your machine.

In order to check that Node.js + npm were successfully install on your machine you can open your Terminal and execute the following command:

npm -v

after typing this command you should see the version of npm that was installed on you machine.

MongoDB

The best place to download and install MongoDB is from MongoDB official site. There are 2 ways to install it: manually or using Homebrew, I recommend to install it via homebrew because it’s much more easy and intuitive.

In order to check that MongoDB is installed on your local machine please open terminal and execute the following command:

mongod

After typing this command you should that MongoDB is starting and waiting for connections on port 27017 which is the default port.

Now that everything is installed on your local machine you are ready to go to the next stage there you will create, run and test your parse-server Node.js app.

Create Your parse-server Application (Practical)

Setup

Open terminal, navigate to where you usually store all your development projects and create a new folder with the name myFirstParseServerApp

mkdir myFirstParseServerApp

navigate to the folder that you just created and create another folder inside it and name it app and finally navigate to the app folder.

cd myFirstParseServerApp/
mkdir app
cd app/

Inside the app folder run npm init in order to initialize a new Node.js application. When you will run this command you will be prompted by npm CLI with questions about the name of the app, the default page and more. Please fill in the relevant details like the app name, description, author name and LICENSE and continue. In my case it looks like the following:

After running the npm init command you will see only one file inside the app folder of your project, the file name is package.json.

package.json is the file which describe your Node.js application. This file contains the name of the app, description, author, version and more. In order to see yourself how this file looks like you can open it with any editor. I personally use Visual Studio Code which is really great and user friendly editor and works very well with Node.js applications. The content of the package.json on my side is:

{
"name": "myfirstparseserverapp",
"version": "1.0.0",
"description": "My first parse-server application",
"main": "index.js",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1"
},
"keywords": [
"parse-server"
],
"author": "Ran Hassid",
"license": "MIT"
}

On your side it should be the same except the name of the author.

Adding parse-server as dependency

The next thing that we will do is to add parse-server module into our app. As i mentioned earlier, parse-server module is hosted on npmjs.org. This allows you to run very simple command in order to add it to your project so lets do that.
In terminal, navigate to the app folder and run the following command:

npm install parse-server --save

After running this command npm will look for parse-server in the npm repository and will install into your project. The flag ( — save) tells npm to also create a dependency and add it to the package.json file. You will also notice that npm will install all the modules which parse-server depends on.

After installation is done go and check your package.json file there you will see that a new dependencies section has been added. The only dependency that we currently have is in parse-server module. Now you probably asking yourself where is the source code that i need? In order to answer on this question you will need to go to your app folder and look for the node_modules library.

Please note! node_modules contain the parse-server library along with all the parse-server dependencies so you will see a lot of folders under it. Please do not change any code in one of those modules because all your changes will be overwritten the next time you will run the npm install command. the node_modules folder must also ignored by your source control (if you use git you need to add to your .gitignore file) because they must be added during installation or deployment time.

Next, create an npm start script that will allow us to run parse server by running the npm start command only. To create the start script add the following:

"start" : "node ./node_modules/.bin/parse-server"

under scripts in your package.json file. After adding the start script your package.json should look like the following:

Next, try to run parse-server on your local machine by executing the following command in terminal (please make sure that you pointing to the app folder):

npm start 

You will immediately notice that your app cannot run because it missing appId and masterKey.

The appId must be added to any request that is being sent to parse-server. It is recommend to create some complicated base64 string as your appId. The masterKey should also be base64 string, using the masterKey parse-server will allow you to run any api call as an admin. The masterKey must be known only to you and the appId must be known only to your application.

You have 2 options to pass those 2 variables to parse-server:

1. Environment Variables — variables that can be accessed in runtime by the Node.js server and are stored under the process.env.{NAME_OF_THE_ENV_VARIABLE} object.
2. Config file — JSON configuration file that will be parsed by parse-server in runtime.

I recommend to use the second option (config file). The reason is that when you use parse-server you will immediately noticed that in addition to appId and masterKey you need also to add more and more configurations for different capabilities that you need and using the first option it will be very hard to understand which of the variables were set and which are not, actually you can only understand it in runtime and in the second option you can understand it in design time (before you running your parse-server) and this is very important.

Create parse-server config.json file

Go to your app folder and create a new config folder under it. Under the config folder create a new config.json file that will contain all the configurations which are required to run your parse-server instance.

Open to config.json file and paste the following content into it:

{
"databaseURI": "mongodb://localhost:27017/my-db",
"appId": "0742vWqaHDnW5VkvbTogyh2O9UbdXDlAe19XOPd2Tc",
"masterKey": "La7U4b8Hxdsu2kwcV6ou08JWGlCdCETjkyR9H8DIXw",
"serverURL": "http://localhost:1337/parse",
"cloud": "./cloud/main.js",
"mountPath": "/parse",
"port": 1337
}

databaseURI — like i’ve mentioned above parse-server use MongoDB to persist the data so the databaseURI is the connection string where your mongoDB is located. Because currently we run it on our local machine we are pointing to localhost and to the default port where MongoDB is running (27017).

appId — the application key that needs to be sent by the client in any request that you sent to parse-server.
masterKey — key which allows you to run api calls as admin please keep it to yourself only.
serverURL — The endpoint for the parse-server api. You will run your api requests in front of this endpoint.
cloud — like i mentioned above, one of the capabilities of parse-server is the ability to run cloud code functions and server side hooks so this parameter will contain the path to the source code of those functions and hooks.
mountPath — the mount path for parse-server. The default is /parse
port — the port where parse-server runs on. The default is 1337

Next, create the file that will contain the source code of your custom cloud code functions and server side hooks:

Under your app folder create a new folder and name it cloud. Under the cloud folder create a new file and name it main.js. The main.js file can be empty for now.

Run parse-server locally

Now it’s time to rerun our parse-server instance again. Go to terminal and run the same command that you ran before but this time send the config.json file.

npm start -- config/config.json

You will noticed that now parse-server is running and listening on port 1337

Troubleshooting: if from some reason you see some errors related to MongoDB please make sure that mongoDB is installed, open a new terminal window in parallel and run mongod command and then rerun your parse-server again.

Test parse-server locally

The best way to test your parse-server instance (from my POV) is with a REST client tool. I personally use POSTMAN which is very intuitive.

If you will try to run a GET request to the following endpoint: http://localhost:1337/parse/classes/user to get the lists of users in your parse-server you will immediately noticed that the response is

{"error":"unauthorized"}

The reason is that the an api key is missing. The api key needs to be added to the request headers so before sending the request make sure that the api key added in the request headers as well.

After rerun the request again you should receive 200 response with an empty result set

{
  "results": []
}

The reason that you are getting empty results is because there are no users in the Database.

In order to create a new user we need to run a POST request to the same URL and pass the required fields in the request payload. The required fields are: username, password, email.

In addition to the required fields we can send more fields. The nice thing here is that you can send as many as fields as you want and that’s because MongoDB is schema-less so you don’t need to create the schema ahead.

The response will from this request will be:

{
  "objectId": "9XL661CuBK",
  "createdAt": "2017-05-16T05:55:37.092Z"
}

parse-server will auto generate the user objectId for you as well as the date that this object was created.

Now, let’s run the same GET request that we ran to get the list of users in our database. After executing the GET request you should get the following response:

{
  "results": [
    {
      "objectId": "9XL661CuBK",
      "username": "ranhsd",
      "password": "init1234",
      "email": "ran@example.com",
      "firstName": "Ran",
      "lastName": "Hassid",
      "createdAt": "2017-05-16T05:55:37.092Z",
      "updatedAt": "2017-05-16T05:55:37.092Z"
    }
  ]
}

Please note! I am sure that part of you are wondering how it is possible to get the list of users without logging in first to the service. The reason is that by default parse-server expect to get only an api key. The problem with api keys is that it isn’t secure enough because anyone who know how to sniff network requests will be able to “steal” your api key easily.

parse-server solve it via ACL’s and class level permissions. I will not cover this topic in this series but ACL’s allows you to define which users/roles can create/update/delete/read which objects. You can read more about it here and here

Containerize parse-server with docker

After running and testing parse-server on your local machine it’s time to run it the right way.

Under the app folder create a new file and name it Dockerfile. This file contain some commands that will be executed when you build/run your docker image.

The content of this file is:

FROM node:boron
RUN mkdir -p /parse-server
WORKDIR /parse-server
COPY . /parse-server
RUN npm install
EXPOSE 1337
CMD ["npm","start"]

Let’s explain:

FROM node:boron — this tells docker that we are running a Node.js app and we need node runtime as dependency. So before creating the image for our app docker will check if a node image is available and if it’s not it will download it from dockerhub and then it will create our image.
RUN mkdir -p /parse-server — Create parse-server directory in the docker image
WORKDIR /parse-server — set the folder that we just created a working directory
COPY . /parse-server — copy all the files from your app into the image working directory
RUN npm install — run the npm install command (on the image) in order to install all the dependencies
EXPOSE 1337 — expose the 1337 port so it will be accessed from outside
CMD [“npm”,”start”] — run npm start to start the parse-server

After creating the Dockerfile we can run a very simple command for building a docker image from our parse-server app. In order to test it please open terminal, navigate to the app folder and execute the following command:

docker build -t my-parse-app .

After executing this command you will notice that docker cli will parse the Dockerfile and will run the commands line by line. The most “heavy” command npm install , this command is the same command that we ran earlier on our machine but this time it runs on the image. In order to validate that the build command worked please run the following command:

docker images

and check that both images (node and my-parse-app) are listed there.

Since our parse-server app depends on mongoDB we will need now to run a mongoDB container, then run our parse-server container and link mongoDB into it. We can do that by first building a mongoDB image, run it via docker run command and then run the parse-server image that we just built with — link flag. Sounds complicated right? That’s where docker-compose comes for rescue.

setup docker-compose

docker-compose allowing us to compose multiple images, build and run them together. The only thing that we need to do is to create a docker-compose.yml file that will list all the services. Using docker-compose we can also:

• run our images with custom commands (that are needed in order to run parse-server with the config.json)
• Mount disk volumes (e.g. for mongoDB files, cloud code files etc.)
• Link multiple containers together easily
• Set environment variables
• and more…

Under the root folder of your project (myFirstParseServerApp) crate a new file and name it: docker-compose.yml. This file will list all the services that we want to build and run with docker.

Since Medium lake support for text indentation and indentation is required in yml files please download it from here or type it yourself (not recommended)

The big advantage of yml files is that they are “human friendly”. You can almost immediately understand from this file that we have the following services listed there:

1. app — the parse-server app linked with the mongo service. Please note that the name of the running container is: my-parse-app
2. mongo — MongoDB database use the public mongoDB image from dockerhub
3. mongodata — mounted by the mongo service and create disk volume where all mongo data will be persisted.

After creating this file your project structure should looks like the following:

Please note that docker-compose.yml is located under the root folder (myFirstParseServerApp) and not under the app folder.

The last thing that we need to do is modify our config.json file specifically the value of databaseURI.

Replace:

mongodb://localhost:27017/my-db

with:

mongodb://mongo:27017/my-db

The reason is that when we run our app with docker the mongoDB image running in another container, so instead of localhost we need to provide the container service name which in our case called mongo

Please note! if you want to run your app in both ways (via docker and locally) you can simply copy and paste your config.json file, change its name (e.g. config-local.json) and use it when you run locally.

run your app

The last thing that left to do is to run and test the app. This time we will run the app via docker-compose. In order to run your app with docker-compose please open terminal, navigate to your project root library (myFirstParseServerApp) and execute the following command:

docker-compose up

After executing this command docker will:

• Download/Reuse the relevant images
• Build your parse-server Node.JS application by executing all the commands in Dockerfile
• Create node, my-parse-app and mongo images and run them.

In order to know which of the containers are currently running on your machine please execute the following command:

docker ps

After executing this command you should see something like this:

Now it’s time to test that everything works as expected. To test it please open POSTMAN and run exactly the same requests that you ran before and if you get the same results it means that everything works as expected.

Summary of part 1

Now, after everything is running well on your local machine in containers you are ready to go to part 2 of this series there I will explain how to deploy and run the same services on google container engine.