Cloudant Fundamentals: Using the Bulk API

Bulk up your CRUD (part 5 of 10)

In the last blog post we saw how to do CRUD (Create/Read/Update/Delete) operations with the Cloudant database using the curl command line tool. In this post we’ll use just two API calls to achieve the same thing, but with the capability of working on multiple documents at the same time.

If you have two or five or a hundred documents to add to Cloudant, then you need to look at the bulk operations API. If your app is going to go through the trouble making an HTTP call (doing a DNS lookup, creating a connection, negotiating a HTTPS handshake, etc.), then it may as well do as much work as it can with that connection while it can. It is much more efficient for you to bulk upload 100 documents in a single bulk request than send them in 100 separate API calls.

There are only two API calls you need to know about:

• GET or POST /db/_all_docs - for reads
• POST /db/_bulk_docs - for creates, updates and deletions

Bulk candy, bulk docs — you get the idea. Photo by Matt Schwartz on Unsplash

Creating documents in bulk

Let’s create a file called bulk.json that contains the documents we want to write:

We can write the three documents in a single API call:

Cloudant replies back with an array, with one element for each document inserted telling you the auto-generated ID and the calculated rev token.

If we’d wanted to specify the _id fields we could have simply included them in the document objects in the submitted bulk.json file.

Reading the documents back in bulk

As well as reading back single documents:

We can use the GET /db/_all_docs endpoint to fetch multiple documents at once if we supply an array of document IDs:

But wait! Where are the document bodies? Cloudant has only returned the ID of the documents (twice!) and the revision token.

If you want the document bodies too, you have specify include_docs=true in your request:

Now we can see the whole document in a doc attribute of each element of the rows array.

Updating documents in bulk

Let’s update our bulk.json file to prepare it for a bulk update. We need to:

• add the _id/_rev of each document
• add the data we want to add, in this case the IMDB URL of each actor

Updating these three documents is simply a matter of posting this JSON to POST /db/_bulk_docs.

And Cloudant gives back a new revision token for each document.

Bulk deletions

The process for bulk deletions is similar to bulk updates, except that we don’t need to supply a document body, only a _deleted: true flag for each _id/_rev pair:

Which we post to _bulk_docs:

Here, we get another set of revision tokens.

Note that you can combine inserts, updates and deletes in the same bulk_docs call. Don’t forget we can use the acurl alias we created in the last post to shorten these commands:

If you’re not keen on command-line tools but want to learn the API, then you could also look at the Postman Chrome extension, which allows low-level API calls to be constructed in a graphical user interface.

Next time

In the next blog we’ll look at the programmatic equivalents of these Cloudant create/read/update/delete and bulk operations.