Shindig: Deployment and DevOps

• Previous article: Shindig: Patterns and Best Practices

My initial deployment strategy for Shindig was to use some nice services that make everything easy for me. I first tried using GrapheneDB for hosting Neo4j and Modulus for Mongo (user accounts) and Meteor. However, this didn’t work our very well for me. Both services had limited hosting options and I couldn’t find any overlapping datacenters between the two services. There was a 100–200ms latency between Metoer and Neo4j and this made my app seem very slow. My app is querying Neo4j quite a bit to update publications reactively so latency with the database is really important for me.

With no other options, I decided to try hosting everything myself. I was quite surprised how easy it was. I installed Meteor alongside Mongo and Neo4j all on the same machine. I got a dual-core Digital Ocean droplet, and everything is working great. I don’t have much DevOps experience so there were a lot of bumps along the way. So I want to detail some notes I took about how I got everything up and running.

Digital Ocean

Head over to Digital Ocean, sign up, and create an Ubuntu Linux droplet. You can give your SSH public key when you create the droplet, but if you forgot to do that, you can simply add it to your authorized keys from the commandline.

cat ~/.ssh/id_rsa.pub | ssh root@<DROPLET_IP> "cat >> ~/.ssh/authorized_keys"

You’ll be prompted for the root password. But after that, you should be able to SSH without a password.

ssh root@<DROPLET_IP>

Once you’re inside the server, you’ll want to install Neo4j.

Neo4j

To install Neo4j, you first need to add a repo to aptitude.

wget -O — http://debian.neo4j.org/neotechnology.gpg.key | apt-key add -
echo 'deb http://debian.neo4j.org/repo stable/' > /etc/apt/sources.list.d/neo4j.list
apt-get update

Then you can install Neo4j.

apt-get install neo4j

Once its done installing, Neo4j should be up and running. You can start, stop, and check the status of Neo4j with the following commands.

service neo4j-service start
service neo4j-service stop
service neo4j-service status

All your Neo4j stuff lives in /var/lib/neo4j. To back up your database, you should be able to just copy /var/lib/neo4j/data.

The default configuration for Neo4j only allows for local connections. But if you want to connect from another server, you’ll need to set up authentication. Although I’m connecting to Neo4j locally, I sometimes connect to the production database from a local Meteor instance so I’d recommend doing this anyways. Simply edit the Neo4j server config:

vim /var/lib/neo4j/conf/neo4j-server.properties

And uncomment the following line:

org.neo4j.server.webserver.address=0.0.0.0

Then navigate to the Neo4j browser console by going to http://<DROPLET_IP>:7474. You’ll be prompted to set an admin username and password.

Verify that you can access Neo4j from your local machine with the authentication you just set up:

curl — user <USERNAME>:<PASSWORD> http://<DROPLET_IP>:7474/db/data/

Now just use that URL to connect to your Neo4j database using ccorcos:neo4j.

Mup

Meteor Up is a sweet tool for deploying your own Meteor projects.

npm install -g mup
mup init
# configure your mup.json
mup setup
mup deploy

Make sure to leave the setupMongo property to true so it will install and configure Mongo for you as well.

You can access the logs from a remote machine using:

mup logs -g

You can deploy as easily as:

mup deploy

And from inside the server, the follow commands are useful:

start <APP_NAME>
stop <APP_NAME>
# tail the logs 
tail -f /var/log/upstart/<APP_NAME>.log
# clear the logs
> /var/log/upstart/<APP_NAME>.log
# mongo shell
mongo <APP_NAME>

Mongo Authentication

Mongo’s default configuration is similar to Neo4j — it only allows local connections. If you want to use the production Mongo instance from your local machine, or host your Mongo database on a separate machine from your Meteor instance, then you’ll need to set up some configurations.

SSH into your droplet and enter the mongo shell for your app.

mongo <APP_NAME>

Then create a user that owns this database.

db.createUser({user:"<NAME>", pwd:"<PASSWORD>", roles: [ { role: "dbOwner", db: "<APP_NAME>"} ]})

Exit the Mongo shell, make sure the app is stopped, and stop the Mongo instance.

stop <APP_NAME>
service mongod stop

Edit the Mongo config file:

vim /etc/mongod.conf

Set the following configurations:

port = <PORT>
bind_ip = <DROPLET_IP>
auth = true
httpinterface = true
setParameter = enableLocalhostAuthBypass=true

Start up your Mongo instance back up:

service mongod start

Check that you can access the mongo console from your local machine and the server:

mongo — host <DROPLET_IP> — port <PORT> -u <NAME> -p <PASSWORD> <APP_NAME>

Set the MONGO_URL environment variable for your Meteor application to:

mongodb://<NAME>:<PASSWORD>@<DROPLET_IP>:<PORT>/<APP_NAME>

And start your Meteor application back up.

start <APP_NAME>

You should be all good to go!

To backup your database, simply copy the contents of /var/lib/mongodb/.

Domain Name Services

Google Domains is by far the nicest service I’ve used for setting up domains. I think the rest suck because they were built in the 90's and Google Domains is relatively new. This stuff can get confusing so I thought I’d help you out.

Basically, just go to Custom resource records, create an A record with the name @ and the IP address in the data field. This will point the root of your domain to your server. You can easily set up a subdomain by putting the subdomain as the name and another IP address as the data.

You’ll also want to create a Subdomain forward Synthetic record that forwards www to the root domain.

So thats its ya’ll. I hope this can help you get up and running on a project quickly and save you from some of the pitfalls I had to deal with. Now go and use the app and let me know what you think!