Identifying Unused EBS Volumes

In the world of cloud computing, unnoticed resources can silently inflate expenses. Among these is the often overlooked unused EBS volume. This article emphasizes the significance of spotting and recovering these dormant assets. Through this, organizations can cut unnecessary costs and enhance cloud efficiency.

Introduction

In this article we will be using AWS’ EventBridge and Lambda functions, we’ll set up a process to regularly check for forgotten EBS volumes in AWS. Our Lambda function will carefully find these overlooked resources, leaving no stone unturned in our quest to save costs. Plus, with SNS, we’ll send out clear reports via email, giving stakeholders the info they need to save money and improve operations. Come along as we explore AWS cost savings, using smart automation to find hidden unused EBS volumes and improve our financial and operational strategies.

Requirements:

• An AWS account.

AWS Services Required:

• IAM
• SNS
• Lambda
• EventBridge

Let’s get started

IAM

We must craft an IAM policy to grant the essential permissions required for this task.

Go to IAM service > Policy > Create policy

• Select “EC2” service from the drop down.

• Now, we need to select the permissions we require. Expand the “List” tab to select the permissions.

• We need permissions to “Describe the EBS volume”. So, select the necessary permissions as shown.

• Select “All” under the Resources option. We need to add additional permission for SNS service too. So, click on “Add more permissions” icon.

• Search for “SNS” to select the service.

• Now select the permissions necessary for SNS service.

• In case you need to monitor the log for this event, you need to add CloudWatch permission as well.

• You can select “All” option in the actions and resources field as it will be easier for us to troubleshoot.

• We’ve added the necessary permissions required for our task. Now you can view the permission by clicking the “JSON” tab on top.

• You can give any name for your policy. Once reviewed, you can proceed to create the policy.

Next, we’ll proceed to create an IAM role and attach the newly created IAM policy to it.

Go to IAM service > Role > Create role

• Choose “Lambda” as the service, as we’re setting up a Lambda function to handle this task.

• Search for the IAM policy we’ve created.

• Please assign a name to your role and proceed with its creation after reviewing.

Creating an unused EBS volume

To accomplish our objective, we'll need to create an EBS volume.

Go to EC2 service > Volumes > Create volume

• Once created, wait for the volume to be in “Available” state

Simple Notification Service(SNS)

AWS SNS is a managed messaging service allowing applications to send and receive notifications via topics, supporting various message types like SMS, email, and HTTP/S endpoints.

In order to send email notifications, we’ll need to create an SNS topic.

Go to SNS service > Select Topics > Create topic

• Choose “Standard” as the type for the topic and provide an appropriate name for it.

• Next you’ll need to create a subscription. Click on “Create subscription” button to do so.

• Select “Email” as the protocol, as we need to receive notifications as email.
• The endpoint is where you’ll specify the email address to receive the notifications. Once, these values are given, you can create the subscription.
• The subscription you’ve created will be in a “pending” state until confirmed.

Please log in to the email address you provided and check for any incoming emails. If the email address was entered correctly, you should have received an email similar to the one below.

• To start receiving notifications, you’ll need to click on the “Confirm subscription” link in the email.

• Upon confirmation, you will receive a message similar to the following.

• Confirm the subscription status by checking it from the AWS console.

LAMBDA

AWS Lambda is a serverless computing service from Amazon Web Services (AWS). It lets you run code without managing servers. You upload your function, and AWS handles scaling and resource management. Lambda functions can be triggered by various AWS services or external events. You only pay for the compute time used.

We’ll develop a Python function to monitor the status of all EBS volumes. It will identify unused volumes and send an email with their volume IDs.

Go to Lambda service > Select Functions > Create functions

• Provide a “Name” for the function.
• I’m selecting “Python 3.9” as our runtime platform.
• Select the IAM role we’ve created earlier.

• Paste the following code for the Lambda function.

import boto3
ec2 = boto3.client('ec2')
sns_client = boto3.client('sns')
volumes = ec2.describe_volumes()

def lambda_handler(event, context):
    unused_volumes = []
    for vol in volumes['Volumes']:
        if len(vol['Attachments']) == 0:
            vols = ("Unused Volume ID = {}".format(vol['VolumeId']))
            unused_volumes.append(vols)
    
    #email
    sns_client.publish(
        TopicArn='arn:aws:sns:ap-southeast-2:079859149351:unused_EBS_notifier',
        Subject='Warning - Unused Volume List',
        Message=str(unused_volumes)
    )
    return "success"

Explanation:

The provided code is a Python Lambda function designed to identify unused EBS volumes and send an email notification with their volume IDs using AWS SNS. Here’s a breakdown of the code:

import boto3

Importing the boto3 library, which provides an interface to interact with AWS services programmatically.

ec2 = boto3.client('ec2')
sns_client = boto3.client('sns')

Creating clients for the EC2 (Elastic Compute Cloud) and SNS (Simple Notification Service) services.

volumes = ec2.describe_volumes()

Using the EC2 client to describe all EBS volumes in the AWS account.

def lambda_handler(event, context):

Defining the Lambda function handler, which is the entry point for the execution of the Lambda function.

unused_volumes = []
    for vol in volumes['Volumes']:
        if len(vol['Attachments']) == 0:
            vols = ("Unused Volume ID = {}".format(vol['VolumeId']))
            unused_volumes.append(vols)

Iterating through each EBS volume retrieved from describe_volumes().

Checking if the volume has no attachments (i.e., it’s unused).

If unused, adding the volume ID to the unused_volumes list.

sns_client.publish(
        TopicArn='arn:aws:sns:ap-southeast-2:079859149351:unused_EBS_notifier',
        Subject='Warning - Unused Volume List',
        Message=str(unused_volumes)
    )

Using the SNS client to publish a message to the specified SNS topic.

The message includes the list of unused volume IDs as its content.

return "success"

Returning “success” to indicate the successful execution of the Lambda function.

• You can verify the functionality of the code by initiating a test run.

The “success” message confirms that the code has executed successfully. Now, let’s check if we’ve received any email notifications.

I’ve received the email containing the following content:

Let’s confirm that the Volume ID we received is correct by copying it from the email and checking it in the AWS console.

Go to EC2 service > Volumes > Paste the copied volume ID and hit “Enter”

Upon verification, the received volume IDs indeed correspond to the unused volume itself.

EventBridge

AWS EventBridge is a serverless event bus service by Amazon Web Services. It enables seamless integration of applications and AWS services by routing events from various sources to targets like AWS Lambda functions, SNS topics, SQS queues, and more. EventBridge supports event-driven architectures, allowing you to automate processes and build scalable applications easily.

We’ll utilize the EventBridge service to schedule a trigger that invokes the Lambda function on every Monday at 9 AM. You can select any time that suits your requirements.

Go to EventBridge service > Schedule > Create schedule

EventBridge will provide information about the next occurrences of the scheduled event. This is helpful as we will get an idea of its recurring pattern.

• Next, choose the Target for the schedule. Since we want to trigger a Lambda function, select “Lambda” and then choose the function we’ve created.

• Proceed with creating the schedule, leaving all other options as default.

We’ve successfully implemented an automated system to monitor and notify the presence of unused EBS volumes in our AWS account.

Conclusion

In summary, we’ve built a reliable system using AWS EventBridge and Lambda functions to regularly scan for forgotten EBS volumes in our AWS account. Our Lambda function works diligently to uncover these unused resources, while SNS sends clear email reports to stakeholders. This proactive approach helps us save costs and improve operations by identifying hidden unused EBS volumes.