On Cloud9: Lambda Development in AWS’s cloud-based IDE
Most of you familiar with AWS will have heard or worked with Lambda, its event-driven compute platform designed to let you run code without provisioning or managing servers. The term “serverless” has been coined to describe this capability, which enables you to focus on building the business logic within your application without worrying about the underlying infrastructure that it runs on. Since its introduction to the AWS ecosystem in early 2015, Lambda has revolutionized the way organizations execute a wide variety of tasks. One of the major driving factors behind its widespread adoption has been the potential for cost reduction as you are only charged for compute time while your code executes. Taking the time to fully invest in Lambda has allowed many organizations to make expensive, under-utilized servers a distant memory.
Although not all applications lend themselves to having AWS Lambda serve as their source of compute power, a class of tasks that are particularly well suited for Lambda are operational style tasks. For example, housekeeping jobs such as the automated shutdown/startup of instances, report generation or backup creation/deletion jobs are all excellent candidates. In the past, these types of tasks have been executed from traditional always-on servers and scheduled using cron type functionality. With the adoption of AWS Lambda, they can be executed in an on-demand manner without the requirement of provisioning and maintaining expensive, always-on compute resources.
Having briefly introduced you to both Lambda and AWS Cloud9, I’m going to provide an example of Lambda function development in AWS Cloud9 by walking through a simple function which creates and deletes Elastic Block Store (EBS) volume snapshots. AWS Cloud9 provides a seamless experience for developing serverless applications, making it easy to run and debug locally before opting for remote execution. Let’s check it out!
AWS Cloud9 Environment Creation
Within the AWS console, you should see Cloud9 as an option under “Developer Tools” in the Services section. After finding the “Create Environment” button, proceed to click through the configuration settings, giving your environment a name and selecting options such as instance type, VPC and Subnet.
IAM Policy and Role Creation
Before proceeding to create a Lambda function in our new Cloud9 environment, we need to define an IAM Policy and IAM role to allow our function to create, delete and modify snapshots and tags. This can be done in the AWS Console by following Services → IAM → Policies → Create Policy
With the policy in place, create an IAM role and attach the newly created policy to it. This IAM role will be used by your Lambda function, providing it with the permissions required to execute successfully.
Lambda Function Creation in AWS Cloud9
This section provides a high-level overview of portions of the code within the snapshot function. If you’re interested in looking at the full solution, it can be found on GitHub at: https://github.com/BruceCutler/snapshot_lambda
Navigate to your Cloud9 developer environment and click “Create Lambda Function”. After giving your function a name, you are presented with a variety of blueprints. I chose to use the ‘hello-world-python’ blueprint with Python2.7 runtime, leaving the trigger as blank. Make sure that you select your newly created IAM role during the function creation process as shown below:
Once the function loads, you’ll probably want to get rid of the pre-populated code, leaving just the lambda_handler function definition statement as shown in Image 4:
The boto3 library is the Amazon SDK for Python, allowing developers to create software that makes use of AWS resources. I make sure to import the library at the top of my code, along with other libraries that will come in useful later in the function. Line 9 below (see Image 5) creates a connection to the low-level client representing EC2, which provides a variety of functions to interface with EC2 objects.
The first step in determining which volumes to snapshot is to gather a list of relevant EC2 instances. This is done by calling the describe_instances function using our EC2 client connection (see line 16 in Image 6). Notice that I can apply filters to my search, narrowing down the result to instances that are either in a running or stopped state.
The describe_instances function from the boto3 library returns a list of instance reservations. A single reservation can contain more than one instance, so I loop through these reservations and store each instance as an item in a list:
Next up is the createSnapshots function (Image 7), which uses helper functions to identify relevant volumes, generate expiration dates, create the snapshots and tag them with the identified expiration date. Here’s a quick explanation of each helper function:
expirationDate: Returns a date on which to expire a new snapshot based on a tag associated with the EC2 instance. In my example, I choose to retain snapshots for production instances for 30 days and all other snapshots for 5 days.
getInstanceVolumes: Returns the EBS volumes for each EC2 instance
addTagsToSnapshot: Adds an ‘Expiration’ tag to the newly created snapshot with the value calculated from the expirationDate function
These helper functions have been excluded from my code samples, but can be found in the full solution on GitHub as noted above:
The following outlines the cleanupSnapshots function (Image 8), which gathers information on snapshots owned by a particular account and reads the expiration tag using the checkExpiration function (not shown). If the expiration date has passed, the snapshot is deleted from the account:
Back in the lambda_handler function (Image 9), the createSnapshots and cleanupSnapshots functions are called within their own try blocks to capture execution errors accordingly:
Collaborative Cloud Development in Cloud9
Collaboration and pair programming are both extremely important within any modern software development project. Utilizing AWS’s user management feature, Identity and Access Management (IAM), Cloud9 makes it easy to share your development environment with other users. Sharing your environment is easily done by navigating to the ‘Collaborate’ tab on the right side (see Image 10)
Clicking on the ‘Share’ button will provide you with a pop-up, allowing you to specify IAM users to share with and the type of access (read/write or read-only) to grant them. Any IAM users who have been granted access will be able to see your development environment listed under the ‘Shared with you’ menu item in Services -> Cloud9 of their AWS Console.
Image 11 shows that the Cloud9 environment has been shared with IAM user ‘bcutler’ with ReadWrite permissions. In addition to the ability to see real time updates from others as they type, a group chat window makes it easy to communicate with co-collaborators. For example, IAM user ‘bcutler’ is actively editing the print statement on line 37 and has posted in the group chat on the right side of the screen.
The above features are simple yet effective, making collaborative cloud development possible for teams of all sizes.
Testing and Deploying Lambda Functions in AWS Cloud9
While Cloud9 provides a clean, snappy editor for the development of Lambda functions, arguably its greatest strength is the ability to execute and debug Lambda functions locally. As you test, you can provide payloads to mock input events commonly used as triggers for functions. The snapshotting Lambda function was designed to run on a schedule, as defined by a CloudWatch Event Rule. The screenshot below (Image 12) shows how an input payload (green and red text) can be used when testing your function:
Another particularly useful feature of the Cloud9 IDE is the ability to set breakpoints within your Lambda function, as shown to the right of Image 12. This allows you to check local variable values as you step through the execution. Finally, Cloud9 provides you with information on memory usage and execution time, helping you to allocate the correct resources and set timeout values before deploying it remotely to your AWS account.
Once you have tested your function thoroughly and are ready to deploy it to your AWS account, the process couldn’t be simpler. In the menu to the right of your developer environment, simply right click on your function listed under local functions and click Deploy. Within a matter of minutes, the latest version of your code will be pushed to your AWS account and visible within Services → Lambda.
Although in its infancy, AWS Cloud9 is a welcome and long overdue addition to the AWS suite of services. Since its inception, the most common complaint about Lambda has concerned difficulties in debugging functions. However, the simple yet effective tools provided by Cloud9 are a clear step in the right direction. Coupled with a streamlined editor and cross collaboration functionality, AWS Cloud9 provides an attractive option for developing your next Lambda function. I’ve only scratched the surface in terms of available functionality, so I urge you to check it out for yourselves. With a few clicks, you too could be developing on Cloud9 in no time!