Custom Indexers for Cloudant

Using JavaScript and Redis for problems that don’t fit Cloudant’s indexing engines

The data

Custom indexes

  • Retrieve the top ten pages viewed
  • Find the number of unique IP addresses used to access our site

Top ten pages

Distinct counts of IP addresses

Streaming the changes

  • Listen to a Cloudant database’s changes feed
  • Update the totals in the Redis database for each change
Basic architecture of a Node.js app connecting Cloudant changes to Redis.

Monthly reporting

Going serverless

A similar architecture, but this time with OpenWhisk connecting Cloudant to Redis.
  • https://openwhisk.ng.bluemix.net/api/v1/web/NAMESPACE/leaderboard/getipcount.json
  • https://openwhisk.ng.bluemix.net/api/v1/web/NAMESPACE/leaderboard/getleaderboard.json

Serverless vs. App

  • OpenWhisk doesn’t allow subscription to the changes feed with "include_docs=true" set, so the OpenWhisk action has to call Cloudant to fetch the document body.
  • OpenWhisk’s stateless nature means that each invocation of the action requires a connection and disconnection to both Cloudant and Redis — the app will reuse the connections again and again.
  • With further refinements to the app, we could reduce the writes to Redis by buffering some of them in the app and only writing to Redis periodically (say every 10 seconds). This approach would be impossible to achieve in OpenWhisk.

Custom indexes vs. Built-in indexes

Go forth and index

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store