How to use CouchDB as Cache Database in your app (3/3)

Redirect to CouchDB records and making backups

Go to part 1: What is CouchDB.

Go to part 2: Basic usage and Rails configuration.

On the final part of this series we are going to see how to redirect requests from Rails to CouchDB records and how we can backup our database.

Caching spreadsheets documents

We come to need to use a Caching Database because the app of one of our clients was taking too long to download some on demand generated spreadsheets reports.

To solve this problem we need to take some issues into consideration:

• We will need to have the user’s reports previously generated so the download process will be faster.
• If the report does not exist we need to generate on demand and store it on the cache database for future downloads.
• During the processing of the data to generate the report the user must be provided with a message that the report is been generated so the system won’t try to generate a report that is processing at that moment.
• If the data of the report on the main database was updated we will need to update the cache database report to reflect the latest information.

Let’s see how we can fulfill all these requirements

Rake task to generate reports

First we need to create a rake task that will generate and store on CouchDB all the reports for the users to be able to download them later on.

We need to generate a file called generate_reports.rb on lib/tasks/ in your rails project. Let’s see some example code for this file:

Rake task to generate all user’s reports

On this rake task we are giving the possibility to pass a group of users ids in case we don’t need to generate all the users reports, we can invoke this task with this line on our terminal:

bundle exec rake couchdb_entries:generate_test_report[12 23 35]

this will generate test_report for users with id: 12, 23 o 35. If we don’t pass any argument to the task it will generate the report for all existing users.

Generating the report

Now for the previous task to work we need a file called reports.rb on app/reports folder of your Rails project. This file should contain a method called generate_test_report. The method will look something like this:

the CouchDB model looks something like this:

the ProcessingReport CouchDB model looks something like:

Fetching report on the Controller

After creating the methods needed to generate the report we need a method on the controller that fetch the report or return a default spreadsheet if the report is in process. On our User controller we should add something like this:

Of course to connect this method with a view we need a new route on config/routes.rb:

Updating report on Cache Database

Assuming that we have a method on our User model that process some data, we need to add some logic so that the user’s report gets updated in the Cache database:

Finally you can visit your page and see if the CouchDB database report is served instead of being generated directly from MySQL.

Backing up CouchDB Database

The most common way of backing up a CouchDB database is to replicate to another instance of CouchDB. Here you can find a detailed visual guide on how to do it.

On this post we are going to see how to do a local backup to a .json compressed file using this tool.

First access your server via ssh console and download the script to where you want to do the backups. Then run this on the command line:

wget https://raw.githubusercontent.com/danielebailo/couchdb-dump/master/couchdb-backup.sh

After that give execution permissions to the file with:

chmod +x couchdb-backup.sh

These are the possible options to use with this script:

Usage: ./couchdb-backup.sh [-b|-r] -H <COUCHDB_HOST> -d <DB_NAME> -f <BACKUP_FILE> [-u <username>] [-p <password>] [-P <port>] [-l <lines>] [-t <threads>] [-a <import_attempts>]
	-b   Run script in BACKUP mode.
	-r   Run script in RESTORE mode.
	-H   CouchDB Hostname or IP. Can be provided with or without 'http(s)://'
	-d   CouchDB Database name to backup/restore.
	-f   File to Backup-to/Restore-from.
	-P   Provide a port number for CouchDB [Default: 5984]
	-u   Provide a username for auth against CouchDB [Default: blank]
	       -- can also set with 'COUCHDB_USER' environment var
	-p   Provide a password for auth against CouchDB [Default: blank]
	       -- can also set with 'COUCHDB_PASS' environment var
	-l   Number of lines (documents) to Restore at a time. [Default: 5000] (Restore Only)
	-t   Number of CPU threads to use when parsing data [Default: nProcs-1] (Backup Only)
	-a   Number of times to Attempt import before failing [Default: 3] (Restore Only)
	-c   Create DB on demand, if they are not listed.
	-q   Run in quiet mode. Suppress output, except for errors and warnings.
	-z   Compress output file (Backup Only)
	-T   Add datetime stamp to output file name (Backup Only)
	-V   Display version information.
	-h   Display usage information.

Example: ./couchdb-backup.sh -b -H 127.0.0.1 -d mydb -f dumpedDB.json -u admin -p password

Backup
To create a compressed backup just run:

bash couchdb-backup.sh -z -b -H 127.0.0.1 -d my-db-name -f dumpedDB.json -u admin -p password

Restore
If you compressed the backup file, first uncompress it with:

gunzip dumpedDB.json.gz

and then run:

bash couchdb-backup.sh -a 5 -l 1000 -r -H 127.0.0.1 -d my-db_name -f dumpedDB.json -u admin_user -p password

Here we specify the number of attempts to restore the database with -a 5 and the number of lines to restore with -l 1000. I recommend using this options to decrease the possibilities of errors with large databases.

Conclusion

In this series we have seen how to use CouchDB as a cache database for your Rails application. I have explained the basic usage of CouchDB, how to configure your Rails application to use it and finally how to back it up. I hope this series have been useful to you, if you have any doubt or need any help to make it work please contact me.

Author: Eng. Alberto Aragón Alvarez

• https://www.linkedin.com/in/albertoaragonalvarez/
• alberto@alturasoluciones.com
• https://twitter.com/aragonalvarez85