Provisioning Neo4j Aura with Terraform

Introduction

Neo4j Aura is a fully-managed service provided by Neo4j available in AWS, Azure, and Google Cloud. There are two types of Neo4j Aura:

1. AuraDB — fully-managed Neo4j graph database
2. AuraDS — fully-managed version of Neo4j Graph Data Science (GDS)

Provisioning Neo4j Aura can performed through its console (https://console.neo4j.io/). From there, you can create/modify/delete either AuraDB or AuraDS instance. However, you might want to automate the provisioning process using IaC tool such as Terraform. Fortunately, Neo4j provides the Aura API that you can integrate into your IaC tool.

In this article, I’m going to share my Terraform script that I use to create and delete Aura instance. The complete source code can be accessed in my GitHub repository: https://github.com/wicaksana/neo4j-aura-terraform.

Getting required Information

Before started, we need to get some information from our Aura console. So, go to Aura console and authenticate. You should see something similar to this:

Neo4j Aura console

Get the tenant ID

One way to get the tenant ID is to inspect the URL parameter tenant in your browser’s address bar:

Tenant id as the URL parameter

Generate Aura API credentials

Next, we need to generate client secret to authenticate against the API. Click on the top right corner (the box containing your username and email) and then click “Account details”:

Then, in the “Account Details” page click “Create” button in the Credentials section:

Give the credential a name and then the ID and secret will be generated for you. Make sure that you make a copy of your secret (either download or copy it) in a safe place because once you close the window, you won’t be able to see it again.

Aura API credentials

Get the valid instance configuration for your tenant

Last information we need is to understand the valid configuration for our instance (e.g. allowed Aura types (professional/enterprise), allowed region, allowed memory size, and so on).

One way is to inspect it through the console by going through the wizard to create new instance and see all allowed values. Another way is to use the API to get this information. I’m going to use the latter. Head to Neo4j Aura API specification page (https://neo4j.com/docs/aura/platform/api/specification/):

Neo4j Aura API specification page

Click “Authorize” and fill in the client id and client secret from previous step. If succeed, you should see the lock icon in the button turns locked:

Authorize button

In this page, you can get any information related to your Aura subscription by calling the correct API path. For our purpose, we will use the GET /tenants/{tenantId} under the “tenants” group.

Click “Try it out” and then fill in your tenant id from previous step, then click “Execute”:

You should see the list of allowed instance configuration for your tenant in the Responses section:

For example above, since the tenant I’m using is from Google Cloud marketplace, I can only spin up Aura instance in Google Cloud environment. It also enforces me to only use Aura Professional type.

Attributes you should take note of from the output above are the following:

1. Cloud provider — valid options can be seen in CloudProvider schema in the Aura API specification page.
2. Memory — Notice the syntax (xGB) e.g. 8GB, 16GB, and so on.
3. Region — the region name. It follows the naming convention used by the cloud provider.
4. Type — Aura type. The complete allowed values for Aura type can be seen in InstanceType schema in the Aura API specification page.
5. version — The Neo4j version of the instance.

Those attributes are needed to run our Terraform job.

A bit about the Terraform script

As of the writing of this article, Neo4j haven’t released the official Terraform provider for Aura. Therefore, I have to resort to use existing REST API provider. I chose mastercard/restapi because of its simplicity to use but it has it’s own drawback that I’ll explain later.

The following is the main script. Firstly, it will perform the authentication. Your client secret and client id is going to be used to generate token valid for certain amount of period. This token will be used to authorize the action you are going to do in your Aura subscription. Secondly, depends on the Terraform command, the script will either create (based on the specification you provided as input variable) or destroy the instance.

Variables script:

And finally, the output script:

Create an Aura instance

Before starting, I’m going to put the credentials as environment variables:

export TF_VAR_client_id=<replace-with-your-client-id>
export TF_VAR_client_secret=<replace-with-your-client-secret>

Let’s create a new Aura instance. You can install terraform locally or in my case I prefer to use Docker (assuming I am in the directory of this source code. The link is provided in the beginning of this article).

Terraform init:

docker run --rm -it \
-e TF_VAR_client_id=$TF_VAR_client_id \
-e TF_VAR_client_secret=$TF_VAR_client_secret \
-w /tf -v $PWD:/tf \
hashicorp/terraform:1.7.1 \
init

Terraform apply:

docker run --rm -it \
-e TF_VAR_client_id=$TF_VAR_client_id \
-e TF_VAR_client_secret=$TF_VAR_client_secret \
-w /tf -v $PWD:/tf \
hashicorp/terraform:1.7.1 \
apply

You’ll immediately asked to fill in some variables like the following:

Input variables when creating the instance

Basically, I want to create a Professional AuraDS instance (Aura type: professional-ds) named “test-tf” (Aura name: test-tf) with size of 8 GB (Aura memory: 8GB ) in GCP (cloud provider: gcp ) within Singapore region (Aura region: asia-southeast1 ). The variables are needed in order to successfully invoke the API endpoint POST /instances .

Suppose that Terraform successfully invoke the endpoint, you’ll get the output describing the details of the new instance as the following. Here you’ll get the password of your instance so take a note and keep it safe somewhere.

Terraform output after successfully apply the instance creation.

Note that the instance creation is done asynchronously so despite Terraform apply has completed but it might take awhile until your instance is ready to use. You can check the progress from the Aura console or by invoking the GET /instances endpoint.

The newly-created instance is up and running as seen in the Aura console.

Deleting the instance

Use terraform destroy to delete the instance:

docker run --rm -it \
-e TF_VAR_client_id=$TF_VAR_client_id \
-e TF_VAR_client_secret=$TF_VAR_client_secret \
-w /tf -v $PWD:/tf \
hashicorp/terraform:1.7.1 \
destroy -input=false

The instance gets deleted.

Limitation & Next Step

As you might have noticed, the script can’t be used to modify the existing instance. This is because the provider (mastercard/restapi) expects PUT method for update method, on the other hands Aura API uses PATCH method for instance update. Improvement can be made by using other REST API provider that offers more flexibility or even writing the provider from scratch. Or if this is tolerable, wait until Aura team releases the official provider.