InfluxDB logical backups

In this blog post you will find out how to backup your InfluxDB database into text files. We call it a logical backup.

As you may know, InfluxDB has the ability to snapshot an instance at a point-in-time. We call it a physical backup in binary format. For this, there is an official tool and documentation.

The difference between a logical and physical backup is obvious: the first one is in plain-text and human-readable format, you get the ability to extract an arbitrary chunk of data “database/retention/measurement(s)” for analysis and manipulation etc.

It is a best practice to employ different types of backups in case something goes wrong. The binary backups tend to corrupt but are much faster, while logical backups are slower but enable incremental backups more efficiently.

At Quiq, we have created a script influx-backup.py to perform a logical type of backup and restore with the following features:

• Use of InfluxDB HTTP API with chunked read/write
• Online backup and restore, no need to stop an instance
• Backup data into text files (InfluxDB line-protocol format)
• Restore from a backup
• Separate file for each measurement
• Backup/restore individual measurements
• Backup/restore a given retention
• Incremental backups using “since”, “until” arguments
• Delayed restore to avoid performance degradation
• Gzip support for backup/restore process

A basic example of dumping a database stats:

# ./influx-backup.py --url https://influxdb.localhost:8086 --user admin --dump --dump-db stats --dir /backups/stats
Password:
>> 2017-11-16 12:50:21 UTC
Starting backup of "stats" db to "/backups/stats" dir
Measurements:
['user_accept_time', 'user_assignments', 'user_response', 'user_status', 'user_utilization']
Measurement fields:
{'user_status': {'new_status': 'string', 'old_status': 'string', 'duration_in_old_status': 'integer'}, 'user_utilization': {'value': 'float'}, 'user_response': {'response_time': 'integer', 'adaptive_response_time': 'integer'}, 'user_assignments': {'amount': 'integer'}, 'user_accept_time': {'accept_time': 'integer'}}
Dumping user_accept_time...17839
Dumping user_assignments...53452
Dumping user_response...17823
Dumping user_status...89422
Dumping user_utilization...107045
Done.
<< 2017-11-16 12:50:50 UTC

And restoring into a new database stats_new:

# ./influx-backup.py --url https://influxdb.localhost:8086 --user admin --restore --restore-db stats_new --dir /backups/stats
Password:
>> 2017-11-16 14:05:30 UTC
Starting restore from "/backups/stats" dir to "stats_new" db.
Files:
['user_accept_time', 'user_assignments', 'user_response', 'user_status', 'user_utilization']
> Confirm restore into "stats_new" db? [yes/no] yes
Loading user_accept_time...17839
Loading user_assignments...53452
Loading user_response...17823
Loading user_status...89422
Loading user_utilization...107045
Done.
<< 2017-11-16 14:06:33 UTC

To do incremental backups you can use the following arguments:

--dump-since DUMP_SINCE    start date in the format YYYY-MM-DD
                           (starting 00:00:00)
--dump-until DUMP_UNTIL    end date in the format YYYY-MM-DD
                           (exclusive)

For example, to backup data for October 2017 it will look like:

—-dump-since 2017-10-01 --dump-until 2017-11-01

I hope this article will help to employ logical backups along with binary ones of your InfluxDB databases.

Thanks for reading!