How to Automate Neo4j Deploys on AWS

David Allen
Feb 20, 2019 · 5 min read

Neo4j already provides some documentation on its site for how to do deployments of Neo4j to common clouds, including AWS. But in this article, I’ll provide sample shell scripts that can do this automatically for you.

These are useful when you want to integrate Neo4j into your CI/CD pipeline and be able to create/destroy instances temporarily, and also just to spin up a sample instance. But really, if you can automate Neo4j deployment, then any other piece of software can make a Neo4j instance whenever it needs, which is extremely handy.

If you have any feedback or questions on these approaches, drop by the Neo4j Community site on this thread and share!

Image for post
Image for post
Neo4j and AWS CloudFormation

Requirements

Before we begin, you’ll need the aws command line interface program, which you can download and install with directions here. The AWS CLI is the main way you can automate all things AWS.

It will also be necessary to generate an access token, and make sure that the environment variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY are defined for you.

CloudFormation

Neo4j provides CloudFormation templates for Neo4j Enterprise standalone (that is, a single database instance), Neo4j Causal Cluster (highly available clusters) and Neo4j Community. So first thing’s first, pick which one you would like to deploy. We’ll cover all three in this article.

CloudFormation is really just a recipe for Amazon Web Services that tells AWS how to deploy a whole set of interrelated resources. By deploying all of this as a stack we can keep all of our resources together, and delete just one thing when we’re done.

In general, Neo4j CloudFormation templates have a few properties worth knowing about before we begin:

Approach

How to deploy is going to be very simple: we just create a new CloudFormation stack, pointing to the right stack template URL to tell AWS what to deploy. We then will give each of our stacks various parameters to control how much hardware we’re using and so on.

We’ll need to specify several common parameters, which you’ll see in the scripts below. Here’s an explanation of what they are.

Let’s get started. Simply run any of these scripts, and it will result in a CloudFormation stack being deployed.

Deploying Neo4j Enterprise Standalone

This will create a single instance of Neo4j without high-availability failover capabilities, but it’s a very fast way to get started.

#!/bin/bashVERSION=3.5.3
export SINGLE_TEMPLATE=http://neo4j-cloudformation.s3.amazonaws.com/neo4j-enterprise-standalone-stack-$VERSION.json
export STACKNAME=neo4j-enterprise-$(echo $VERSION | sed s/[^A-Za-z0-9]/-/g)export INSTANCE=r4.large
export REGION=us-east-1
export SSHKEY=my-ssh-keyname
aws cloudformation create-stack \
--stack-name $STACKNAME \
--region $REGION \
--template-url $SINGLE_TEMPLATE \
--parameters ParameterKey=InstanceType,ParameterValue=$INSTANCE \
ParameterKey=NetworkWhitelist,ParameterValue=0.0.0.0/0 \
ParameterKey=Password,ParameterValue=s00pers3cret \
ParameterKey=SSHKeyName,ParameterValue=$SSHKEY \
ParameterKey=VolumeSizeGB,ParameterValue=37 \
ParameterKey=VolumeType,ParameterValue=gp2 \
--capabilities CAPABILITY_NAMED_IAM

Deploying Neo4j Enterprise Causal Cluster

When you need a clustered deploy, you want to use the Causal Cluster template. Note that the major difference here from the last one is the ClusterNodes parameter, where you can pick any number from 3 on up to indicate how many core nodes you want in your cluster.

#!/bin/bashVERSION=3.5.3
export CLUSTER_TEMPLATE=http://neo4j-cloudformation.s3.amazonaws.com/neo4j-enterprise-stack-$VERSION.json
export STACKNAME=neo4j-enterprise-$(echo $VERSION | sed s/[^A-Za-z0-9]/-/g)export INSTANCE=r4.large
export REGION=us-east-1
export SSHKEY=my-ssh-keyname
export CORES=3
export READ_REPLICAS=1
aws cloudformation create-stack \
--stack-name $STACKNAME \
--region $REGION \
--template-url $CLUSTER_TEMPLATE \
--parameters \
ParameterKey=ReadReplicas,ParameterValue=$READ_REPLICAS \
ParameterKey=ClusterNodes,ParameterValue=$CORES \ ParameterKey=InstanceType,ParameterValue=$INSTANCE \
ParameterKey=NetworkWhitelist,ParameterValue=0.0.0.0/0 \
ParameterKey=Password,ParameterValue=s00pers3cret \
ParameterKey=SSHKeyName,ParameterValue=$SSHKEY \
ParameterKey=VolumeSizeGB,ParameterValue=37 \
ParameterKey=VolumeType,ParameterValue=gp2 \
--capabilities CAPABILITY_NAMED_IAM

Deploying Neo4j Community

Using Neo4j community is barely any different than using Neo4j Enterprise (in terms of the deploy process) other than the different CloudFormation stack URL.

#!/bin/bashVERSION=3.5.3
export COMMUNITY_TEMPLATE=http://neo4j-cloudformation.s3.amazonaws.com/neo4j-community-standalone-stack-$VERSION.json
export STACKNAME=neo4j-comm-$(echo $VERSION | sed s/[^A-Za-z0-9]/-/g)export INSTANCE=r4.large
export REGION=us-east-1
export SSHKEY=my-ssh-keyname
aws cloudformation create-stack \
--stack-name $STACKNAME \
--region $REGION \
--template-url $COMMUNITY_TEMPLATE \
--parameters ParameterKey=InstanceType,ParameterValue=$INSTANCE \
ParameterKey=NetworkWhitelist,ParameterValue=0.0.0.0/0 \
ParameterKey=Password,ParameterValue=s00pers3cret \
ParameterKey=SSHKeyName,ParameterValue=$SSHKEY \
ParameterKey=VolumeSizeGB,ParameterValue=37 \
ParameterKey=VolumeType,ParameterValue=gp2 \
--capabilities CAPABILITY_NAMED_IAM

Checking to see if your instance is up

In each case, the commands above simply submit a CloudFormation stack to be deployed, they don’t wait until the stack is available. If you want to do this (so that you can trigger some next step in a software pipeline) you’ll want to wait for the CloudFormation stack to finish deploying, like so:

aws cloudformation wait stack-create-complete --region $REGION --stack-name "$STACKNAME"

This call will block until you’re finished deploying. Finally, you’ll want to get the stack outputs, like this:

aws cloudformation describe-stacks --region $REGION --stack-name "$STACKNAME"

In general this will output a lot of ugly JSON. To cut straight to the outputs of the stack, use the jq tool. By piping the output through:

jq -r '.Stacks[0].Outputs[]'

You’ll get a nice set of outputs and be able to spot the IP address and password of your new instance. By the time the CloudFormation template is finished deploying, the service will be live and ready to go!

Cleaning up and Removing Your Stack

When you’re done with your CloudFormation stack, you can delete it similarly. I use a script like this, and simply pass it the stack name that I created above as an argument to the script.

Keep in mind you must specify the region correctly.

#!/bin/bashecho "Deleting stack $1"
aws cloudformation delete-stack --stack-name "$1" --region us-east-1

That’s it!

Neo4j Developer Blog

Developer Content around Graph Databases, Neo4j, Cypher…

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store