How To Automate AWS EBS Snapshot Using AWS Snapshot Lifecycle Policy

AWS EBS volume’s snapshots are a way to achieve Disaster Recovery. Also a low cost backup method for your entire virtual machine. It’ll do incremental copies. If you are here reading this, I assume you already know what snapshot is and how it works — so I will not bore you further with more details. Let’s see what’s new with this snapshot.

In earlier days, we were using shell scripts or API calls to take the snapshots. There was no native way to automate snapshots and using scripts meant that your scripts could fail and you had to monitor them as well. Later AWS introduced Lambda, a serverless services which to a certain extent solved these problems. But you still had to come up with code for the snapshot lambda function or use an existing one from the internet :). But it all changed yesterday.

AWS introduced a new feature called Snapshot Lifecycle Policy to automate the snapshot process. Few months back they introduced EC2 auto start and stop. Let’s start play with this.

How this works?

To take the snapshot, any API calls needs necessary permission. Here AWS created a default role called AWSDataLifecycleManagerServiceRole for this with the following policy.

{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ec2:CreateSnapshot",
"ec2:DeleteSnapshot",
"ec2:DescribeVolumes",
"ec2:DescribeSnapshots"
],
"Resource": "*"
},
{
"Effect": "Allow",
"Action": [
"ec2:CreateTags"
],
"Resource": "arn:aws:ec2:*::snapshot/*"
}
]
}

It has to be able to create the snapshot on all the regions.

This will take the snapshots based on the tags. So all of your EBS volumes(which all are needs to be take the snapshot) must be tagged before enable this policy.

Limitations:

  • You have to wait for an hour after created the policy.
  • If anyone of your tags are already a part of any lifecycle policy, then you can’t use the same tag for another policy. Else it’ll show the below error.
The following target tag(s) are already in use: {[Key=Env,Value=Dev]}
  • If you removed your policy or changed the tags from the policy, the snapshots which all are taken previously from the snapshot will not be deleted. Still you can use them.
  • If you copy the snapshot which is taken by the policy, it won’t be coming under automated deletion. You have to remove them manually.
  • If you deleted the EBS volumes, the snapshots which are then from policy will not be deleted until the retention policy.
  • You can create snapshot at every 12hr or every 24hr only.
  • The max retention period per volume is 1000. If you give more days then it’ll automatically remove after 1000 days and keep the new snapshots.

Create the Snapshot lifecycle policy:

  1. Go to EC2 console.

2. Under the Elastic Block Store, you can see the Lifecycle Manager.

3. Click Create snapshot policy.

  • Description: Give a name for your policy.
  • Target volumes with tags: Type your tag [ Key : Value], or simply select it from the drop down list.
  • Schedule name: Give a name for your schedule.
  • Create snapshots every: Choose 12hr or 24hr.
  • Snapshot creation start time: Set the start time in UTC.
  • Retention rule: Set the retention period in number.
  • Tag created snapshots: If need you can set the Tag for the snapshot. But by default the below tag are going to be applied.
    aws:dlm:lifecycle-policy-id
    aws:dlm:lifecycle-schedule-name
  • IAM role: It’ll automatically choose the default role, if need you can assign your own roles.
  • Policy status after creation: Select ENABLE if the policy needs to enable immediately after creating the policy or choose DISABLE, so you can enable it later.

Have some fun:

I have noticed that there is no validation on the retention period box, Even no popups to mention the max retention period days. So what will happen if we give more than 1000?

Lets start with 999999999

Then 999999999999…………forgot the count……9

NOTE:

EBS snapshots and AMI are not recommended DR or backup solution for Database server’s data volumes. So use native database backups to take backup for databases.

Hope, some of you are going to move away from old scripts and lambda functions and move to this native way. Happy snapshotting! :)