Cloudant and CouchDB Replication with couchreplicate

A new command-line tool for managing and monitoring multiple replication jobs

Published in

Center for Open Source Data and AI Technologies

3 min readFeb 22, 2018

One of Apache CouchDB’s killer features is replication. JSON data is easily replicated between data centers, from a mobile device to the cloud, or vice versa. The CouchDB replication protocol is shared by Apache CouchDB™ itself, the IBM Cloudant database-as-a-service, Cloudant Sync libraries for iOS and Android and the PouchDB in-browser database.

Setting up a single replication is as easy as filling in a form in the Replication tab of your Cloudant or CouchDB dashboard:

Configuring a replication job in the Cloudant dashboard.

But what if you have hundreds of databases to replicate? What if you want to move lots of small databases, but only want to replicate 5 at any one time? This is where couchreplicate is here to help.

couchreplicate is a command-line tool that allows you to kick off and monitor replications with minimal effort.

Installing couchreplicate

Assuming you already have Node.js and npm installed on your machine, couchreplicate is installed with a single line:

$ npm install -g couchreplicate

You should now have the couchreplicate tool installed on your machine. Check with:

$ couchreplicate --help

Replicating a single database

Migrating one database is as easy as running couchreplicate, supplying your source URL and target URL.

$ couchreplicate -s http://u:p@localhost:5984/mysource -t https://U:P@HOST.cloudant.com/mytarget
food [▇▇▇▇▇▇▇▇———————————————————————————] 32% 21.1s triggered

Your replication will start and you should see a progress bar start to fill up. You can also supply the database name as a separate -d parameter:

$ couchreplicate -d food -s http://u:p@localhost:5984 -t https://U:P@HOST.cloudant.com

And that database name will be used at both the source and target ends.

Replicating multiple databases

The tool really comes into its own when replicating more than one database. Simply supply a comma-separated list of database names in your -d parameter:

$ couchreplicate -d food,drink,hardware,software -s http://u:p@localhost:5984 -t https://U:P@HOST.cloudant.com
food  [▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇]100% 42.5s completed
drink [▇▇▇▇▇▇▇▇———————————————————————————] 32% 21.1s triggered

As one replication completes, another will be scheduled, until the entire list is exhausted.

You can control the number of simultaneous replications with the -c parameter:

$ couchreplicate -c 3 -d food,drink,hardware,software -s http://u:p@localhost:5984 -t https://U:P@HOST.cloudant.com
food     [▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇—————] 82% 34.5s triggered
drink    [▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇———————————————] 63% 34.1s triggered
hardware [▇▇—————————————————————————————————————] 10% 34.1s triggered

Replicating all of your databases

Instead of supplying a list of database names, the -a option can be used to indicate that all of the databases in the source cluster are to be migrated. This is handy if you are moving from local CouchDB to Cloudant, or from one Cloudant service to another.

$ couchreplicate -c 3 -a -s http://u:p@localhost:5984 -t https://U:P@HOST.cloudant.com

It’s all about the open source

The couchreplicate tool is free to use and is open-sourced under the Apache 2.0 license, so you can fork the code yourself and modify as you see fit. If you find a bug or have a contribution, we’d love to hear from you on the project’s Github page.