Moving data from DocumentDB to Cloudant or CouchDB

Introducing the documentdbexport npm module

In my last blog, we looked at moving data from Amazon DynamoDB to Cloudant or CouchDB. In this article we’re going to look at extracting data from Microsoft Azure’s DocumentDB service.

You’ll use the documentdbexport npm module to extract data, and the couchimport module to import it, of course.

Getting the data out

Once again, I’ve written a script to do this for you: documentdbexport.

First, install the tool:

$ npm install -g documentdbexport

Define a couple of environment variables with your Azure credentials:

$ export AZURE_ENDPOINT="https://MYDOCDB.documents.azure.com:443/"
$ export AZURE_KEY="GeIZysnonvgpk2"

Then, simply run documentdbexport, supplying the name of the database and collection to export:

$ documentdbexport --database iot --collection temperaturereadings
{"temperature":30730,"time":"2017-03-09T02:21:48+0000","_id":"1489026108"}
{"temperature":17072,"time":"2017-03-09T02:15:22+0000","_id":"1489025722"}
{"temperature":18177,"time":"2017-03-08T21:27:23+0000","_id":"1489008443"}
Export complete { records: 3, time: 0.145 }

The tool makes as many API calls as it needs to extract the data, converting the JSON to a more compact form as it goes.

Importing into CouchDB/Cloudant

We can use couchdbimport to do the import stage for us. Install it with:

$ npm install -g couchimport

Set an environment variable with your target Cloudant/CouchDB service’s URL:

$ export COUCH_URL="https://MYUSER:MYPASS@MYHOST.cloudant.com"

Then, run both the documentdbexport and couchimport commands together, piping the output of the former into the latter:

$ documentdbexport --database iot --collection temperaturereadings | couchimport --db iot --type jsonl

The --type jsonl parameter tells couchimport that it is to expect one JSON document per line and --db iot defines the name of the target database.

It’s that simple! You’ll find more details on command-line usage and programmatic access for documentdbexport on npm. And please ♡ this article if you’d like to recommend it to other Medium readers.