AWS EBS Volumes and Monitoring

6 min readJan 17, 2017

Understanding EBS Volumes types and Monitoring

EBS Stands for Elastic Block Store (Amazon EBS) is basically Elastic Block Storage for EC2 instances. It provides block level storage for use of ec2 instances. EBS volumes are highly available and reliable storage volumes that can be attached to running instance within same availability zone.

EBS Volumes that are attached to ec2 instance are like storage volumes. which helps in retaining volumes even if you stop ec2 instances.

What are the different types of EBS Volumes ?

It comes in 3 different types of EBS Storage

Magnetic (st1 and sc1)
General Purpose SSD (GP2)
Provisioned IOPS SSD (io1)

Now lets try to understand by comparing them From the below table.

IOPS stands for Input/Output Operation Per Second. To know more about it read my other blog.

Lets understand how to calculate iops performance for General Purpose. It says 3 IOPS/GB which means if i take 100GB disk i will get 3*100=300 IOPS as base line performance. But there is a catch here he also says brust of 3000 IOPS.Means when you server has more io to do it can increase 3K IOPS. But how long ? So has a credit system.

What are I/O credits ? How do we get them ?

Each volume receives an initial I/O credit balance of 5.4 million I/O credits, which is enough to sustain the maximum burst performance of 3,000 IOPS for 30 minutes.

For example, a 100 GiB gp2 volume has a baseline performance of 300 IOPS.When your volume requires more than the baseline performance I/O level, it draws on I/O credits in the credit balance to burst to the required performance level, up to a maximum of 3,000 IOPS.

Volumes larger than 1,000 GiB have a baseline performance that is equal or greater than the maximum burst performance, and their I/O credit balance never depletes. When your volume uses fewer I/O credits than it earns in a second, unused I/O credits are added to the I/O credit balance.

The maximum I/O credit balance for a volume is equal to the initial credit balance (5.4 million I/O credits).

I/O Credits and Burst Performance

The performance of gp2 volumes is tied to volume size, which determines the baseline performance level of the volume and how quickly it accumulates I/O credits; larger volumes have higher baseline performance levels and accumulate I/O credits faster. I/O credits represent the available bandwidth that your gp2 volume can use to burst large amounts of I/O when more than the baseline performance is needed. The more credits your volume has for I/O, the more time it can burst beyond its baseline performance level and the better it performs when more performance is needed. Below example will be more clear.

Let'sdo some calculations and understand how much brust I can get ?

Example1:

Say we have 1GB Volume and we get 3 IOPS per 1GB so we have 3*1=3

Max brust for GP1 is 3000 IOPS and for 1GB and using IO credit it will be 3000–3=2997 IOPS

Example2:

Say we have 200GB Volume and we get 3 IOPS per 1GB so we have 3*200=600

Max brust for GP1 is 3000 IOPS .

Using IO credit it will be 3000–600=2400 IOPS

Example3:

If we need to get 10000 IOPS how much disk I should have ?

We need 3333GB disk. ie 3*3333=9999GB

Bottom line

You need to choose right disk space to get right IOPS. And you need to remember you only get 3000 IOPS and if you need more increase disk space.

First Time EBS Volume Access

EBS Volumes are persistent block level storage. All storage volumes are immediately allocated however when we access first time EBS volumes either created new volume or initiated restore from snapshot it will have a loss of 10%–50% IOPS. Credits will be taken from initial allocation of 5.4 million credits.Also there will be performance hit for first time access of EBS volumes and might impact be bad user experience. To avoid this we need to perform Pre-Warming of EBS Volumes.

Pre-Warming New EBS Volumes:

Identify your EBS volume.
In order to ensure that you are pre-warming the right volume, please use the lsblk command to list your EBS volumes. Select the one that needs to be pre-warmed.

# lsblk

Before you proceed with pre-warming, ensure that your EBS volume is not mounted. If it is, use the command below to unmount it:

# umount <mount-point>

For example, if your volume is mounted on folder called ‘mount point’, you can use the command below:

Pre-warm your EBS volume
In order to pre-warm an EBS volume on your Linux instance, you can use the following dd command:

CAREFUL: It is for new disks. Don't do it if you are restoring from snapshot

# dd if=/dev/zero of=/dev/xvdX bs=1M

This operation might take time depending on the size of the volume. Once complete, however, your EBS volume will be pre-warmed and ready for use.

Pre-Warm an EBS Volume from a Snapshot — Linux

If you want to pre-warm an EBS volume that was restored from an EBS snapshot, you have to use a different approach than the ones outlined above. As mentioned earlier, the approaches above will basically erase all of the data on your newly created EBS snapshot. In order to pre-warm an EBS volume that was restored from an EBS snapshot, you can leverage the dd command to:

Pre-warm existing data in which case dd will read and pre-warm your existing data from snapshots. Because this is a read operation, it won’t pre-warm unused space from restored snapshot.

In order to perform this operation, use the dd command to read blocks that contain data and write on them using ‘/dev/null’. This will only pre-warm existing data blocks. The syntax to perform this is:

# sudo dd if=/dev/xvdX of=/dev/null bs=1M

For example:

Pre-warm existing data and un-used space with a read and write back operation where a command reads your blocks and writes them back to the same location. This pre-warms your complete volume and existing data won’t be erased.

You can use the following command:

# sudo dd if=/dev/xvdX of=/dev/xvdX conv=notrunc bs=1M

For example: