Cloudant Fundamentals: Using the Bulk API

Bulk up your CRUD (part 5 of 10)

Glynn Bird
Jun 4, 2018 · 3 min read

In the last blog post we saw how to do CRUD (Create/Read/Update/Delete) operations with the Cloudant database using the curl command line tool. In this post we’ll use just two API calls to achieve the same thing, but with the capability of working on multiple documents at the same time.

If you have two or five or a hundred documents to add to Cloudant, then you need to look at the bulk operations API. If your app is going to go through the trouble making an HTTP call (doing a DNS lookup, creating a connection, negotiating a HTTPS handshake, etc.), then it may as well do as much work as it can with that connection while it can. It is much more efficient for you to bulk upload 100 documents in a single bulk request than send them in 100 separate API calls.

There are only two API calls you need to know about:

  • GET or POST /db/_all_docs - for reads
  • POST /db/_bulk_docs - for creates, updates and deletions
Image for post
Image for post
Bulk candy, bulk docs — you get the idea. Photo by Matt Schwartz on Unsplash

Creating documents in bulk

Let’s create a file called bulk.json that contains the documents we want to write:

We can write the three documents in a single API call:

Cloudant replies back with an array, with one element for each document inserted telling you the auto-generated ID and the calculated rev token.

If we’d wanted to specify the _id fields we could have simply included them in the document objects in the submitted bulk.json file.

Reading the documents back in bulk

As well as reading back single documents:

We can use the GET /db/_all_docs endpoint to fetch multiple documents at once if we supply an array of document IDs:

But wait! Where are the document bodies? Cloudant has only returned the ID of the documents (twice!) and the revision token.

If you want the document bodies too, you have specify include_docs=true in your request:

Now we can see the whole document in a doc attribute of each element of the rows array.

Updating documents in bulk

Let’s update our bulk.json file to prepare it for a bulk update. We need to:

  • add the _id/_rev of each document
  • add the data we want to add, in this case the IMDB URL of each actor

Updating these three documents is simply a matter of posting this JSON to POST /db/_bulk_docs.

And Cloudant gives back a new revision token for each document.

Bulk deletions

The process for bulk deletions is similar to bulk updates, except that we don’t need to supply a document body, only a _deleted: true flag for each _id/_rev pair:

Which we post to _bulk_docs:

Here, we get another set of revision tokens.

Note that you can combine inserts, updates and deletes in the same bulk_docs call. Don’t forget we can use the acurl alias we created in the last post to shorten these commands:

If you’re not keen on command-line tools but want to learn the API, then you could also look at the Postman Chrome extension, which allows low-level API calls to be constructed in a graphical user interface.

Next time

In the next blog we’ll look at the programmatic equivalents of these Cloudant create/read/update/delete and bulk operations.

Center for Open Source Data and AI Technologies

Things we made with data at IBM’s Center for Open Source…

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store