AWS services (EBS, RDS, EFS, Elasticsearch) backup solutions
A whitepaper on backup and restore approaches using AWS
EBS
EBS snapshot (manual, on-demand)
Snapshot Lifecycle Manager : Allows to automate the creation (scheduling), retention (no. of days), and deletion of Amazon EBS volume snapshots. Powered by tags, means, the policies apply only for volumes having defined set of tags.
Using CloudWatch events : CloudWatch Event rules can be created to schedule EBS snapshots.
AWS Ops Automator : To easily configure schedules to automatically create, copy, and delete Amazon Elastic Block Store (Amazon EBS) snapshots using CloudFormation Template (that uses Lambda function)
Using Lambda functions : Lambda functions can be scheduled to run using CloudWatch events, or other supported events.
Python3 supported code for above lambda function
Lambda function to create AMI (as backup)
RDS
RDS backup options are mentioned here.
For Database services like RDS and Neo4j, snapshots is a possible solution. However, Database backups could be final solution that could be done via scripts.
This story covers mainly RDS Postgre SQL and Aurora Postgre SQL backup options.
Automated backups
- To backup databases and transaction logs
- Storage volume snapshots are created for the backup, which have retention period.
- This configuration is done during DB instance creation, and can be modified later.
- For PostgreSQL DB instance : Automated backups (0–35 days retention. 0 days = no backup) + “Read Aurora replica”
- For Aurora PostgreSQL DB instance : Only Automated backups (1–35 days retention i.e. Automated backup is must).
- More info on Aurora DB clustering (single endpoint, R/W, LB etc.)
Manual/On-demand backups (snapshots)
- Take EBS snapshot (Snapshots are incremental backups — which is then stored into S3 as object), which can be restored later.
- Works for both PostgreSQL and Aurora PostgreSQL
Always New DB Instance is created on Restore, either with “Snapshot” or “Native Automated Backup” or “AWS Backup”. Workaround (because DB instance should have unique name) : Rename original DB instance -> Restore -> New DB instance can have original name
For Postgre SQL DB instance, after “snapshot restore”, the Databases from the original DB instance are gone, not accessible. All are accessible in the restored DB instance.
As of now, “Restore from S3” option is available only for MySQL and Aurora MySQL.
EFS
EFS backup options are mentioned here.
Automatically creates incremental backups of an Amazon EFS file system on a customer-defined schedule Using CloudFormation Template
Copies data from Amazon EFS file system (called the production file system) to another Amazon EFS file system (called the backup file system).
Automated EFS scheduled backups (for example, hourly, daily, weekly, or monthly).
Automated rotation of the backups, where the oldest backup is replaced with the newest backup based on the number of backups that you want to retain.
Quicker backups using rsync, which only back up the changes made between one backup to the next.
Elasticsearch service
ES Backups includes a cluster’s data and state that includes cluster settings, node information, index settings, and shard allocation.
Automatic snapshot configuration
ElasticSearch cloud-aws plugin
Uses ElasticSearch’s snapshot (stored on S3) and restore functionalities.
Also, allows the cluster to auto-discover new members of the cluster when they turn-on.
As listed above, there are several options available for backup and restore of different AWS services.
Additionally, AWS has a dedicated service called AWS Backup to centrally manage backups of multiple services.
