On Cloud9: Lambda Development in AWS’s cloud-based IDE

Most of you familiar with AWS will have heard or worked with Lambda, its event-driven compute platform designed to let you run code without provisioning or managing servers. The term “serverless” has been coined to describe this capability, which enables you to focus on building the business logic within your application without worrying about the underlying infrastructure that it runs on. Since its introduction to the AWS ecosystem in early 2015, Lambda has revolutionized the way organizations execute a wide variety of tasks. One of the major driving factors behind its widespread adoption has been the potential for cost reduction as you are only charged for compute time while your code executes. Taking the time to fully invest in Lambda has allowed many organizations to make expensive, under-utilized servers a distant memory.

While the functionality provided by Lambda has been positively received by AWS users, a common complaint since its introduction has concerned difficulty in debugging Lambda functions. However, given that 90–95% of AWS’s new product offerings come as a direct result of customer or partner feedback, the recent release of AWS Cloud9 comes as no surprise. Announced at re:Invent 2017, AWS Cloud9 is an on-demand, cloud-based Integrated Development Environment (IDE) that lets you write, run and debug code in your web browser. In addition to standard editing and debugging tools for a number of languages (JavaScript, Python, PHP, Ruby, Go and C++), it features a host of AWS integrations and a slick suite of pair programming tools to make collaborative cloud development easy.

Although not all applications lend themselves to having AWS Lambda serve as their source of compute power, a class of tasks that are particularly well suited for Lambda are operational style tasks. For example, housekeeping jobs such as the automated shutdown/startup of instances, report generation or backup creation/deletion jobs are all excellent candidates. In the past, these types of tasks have been executed from traditional always-on servers and scheduled using cron type functionality. With the adoption of AWS Lambda, they can be executed in an on-demand manner without the requirement of provisioning and maintaining expensive, always-on compute resources.

Having briefly introduced you to both Lambda and AWS Cloud9, I’m going to provide an example of Lambda function development in AWS Cloud9 by walking through a simple function which creates and deletes Elastic Block Store (EBS) volume snapshots. AWS Cloud9 provides a seamless experience for developing serverless applications, making it easy to run and debug locally before opting for remote execution. Let’s check it out!


AWS Cloud9 Environment Creation

Within the AWS console, you should see Cloud9 as an option under “Developer Tools” in the Services section. After finding the “Create Environment” button, proceed to click through the configuration settings, giving your environment a name and selecting options such as instance type, VPC and Subnet.

Image 1: AWS Cloud9 Environment Creation

After clicking the final “Create Environment” button, your development environment will be available for use within 1 or 2 minutes. Once active, you can configure themes and keyboard modes to meet your functional and aesthetic needs. At the bottom of the screen, notice that you have command prompt access to the underlying Linux EC2 resource powering your Cloud9 development environment. With the AWS Command Line Interface (CLI) pre-installed, this feature can be useful for quickly obtaining information on resources within your AWS account. In addition, the Javascript REPL allows you to test single or multi-line code in a browser like iframe.

Image 2: AWS Cloud9 Environment Welcome Screen

IAM Policy and Role Creation

Before proceeding to create a Lambda function in our new Cloud9 environment, we need to define an IAM Policy and IAM role to allow our function to create, delete and modify snapshots and tags. This can be done in the AWS Console by following Services → IAM → Policies → Create Policy

IAM Policy for Snapshots and Tags

With the policy in place, create an IAM role and attach the newly created policy to it. This IAM role will be used by your Lambda function, providing it with the permissions required to execute successfully.


Lambda Function Creation in AWS Cloud9

This section provides a high-level overview of portions of the code within the snapshot function. If you’re interested in looking at the full solution, it can be found on GitHub at: https://github.com/BruceCutler/snapshot_lambda

Navigate to your Cloud9 developer environment and click “Create Lambda Function”. After giving your function a name, you are presented with a variety of blueprints. I chose to use the ‘hello-world-python’ blueprint with Python2.7 runtime, leaving the trigger as blank. Make sure that you select your newly created IAM role during the function creation process as shown below:

Image 3: AWS Lambda function setup

Once the function loads, you’ll probably want to get rid of the pre-populated code, leaving just the lambda_handler function definition statement as shown in Image 4:

Image 4: Blank Lambda Function Handler

The boto3 library is the Amazon SDK for Python, allowing developers to create software that makes use of AWS resources. I make sure to import the library at the top of my code, along with other libraries that will come in useful later in the function. Line 9 below (see Image 5) creates a connection to the low-level client representing EC2, which provides a variety of functions to interface with EC2 objects.

Image 5: Initial Imports and EC2 Connection Creation using boto3

The first step in determining which volumes to snapshot is to gather a list of relevant EC2 instances. This is done by calling the describe_instances function using our EC2 client connection (see line 16 in Image 6). Notice that I can apply filters to my search, narrowing down the result to instances that are either in a running or stopped state.

The describe_instances function from the boto3 library returns a list of instance reservations. A single reservation can contain more than one instance, so I loop through these reservations and store each instance as an item in a list:

Image 6: Searching for reservations and collecting instances

Next up is the createSnapshots function (Image 7), which uses helper functions to identify relevant volumes, generate expiration dates, create the snapshots and tag them with the identified expiration date. Here’s a quick explanation of each helper function:

expirationDate: Returns a date on which to expire a new snapshot based on a tag associated with the EC2 instance. In my example, I choose to retain snapshots for production instances for 30 days and all other snapshots for 5 days.

getInstanceVolumes: Returns the EBS volumes for each EC2 instance

addTagsToSnapshot: Adds an ‘Expiration’ tag to the newly created snapshot with the value calculated from the expirationDate function

These helper functions have been excluded from my code samples, but can be found in the full solution on GitHub as noted above:

Image 7: createSnapshots function

The following outlines the cleanupSnapshots function (Image 8), which gathers information on snapshots owned by a particular account and reads the expiration tag using the checkExpiration function (not shown). If the expiration date has passed, the snapshot is deleted from the account:

Image 8: cleanupSnapshots function

Back in the lambda_handler function (Image 9), the createSnapshots and cleanupSnapshots functions are called within their own try blocks to capture execution errors accordingly:

Image 9: Calling createSnapshots and cleanupSnapshots within the handler

Collaborative Cloud Development in Cloud9

Collaboration and pair programming are both extremely important within any modern software development project. Utilizing AWS’s user management feature, Identity and Access Management (IAM), Cloud9 makes it easy to share your development environment with other users. Sharing your environment is easily done by navigating to the ‘Collaborate’ tab on the right side (see Image 10)

Image 10: Share your environment using the Collaborate Tab

Clicking on the ‘Share’ button will provide you with a pop-up, allowing you to specify IAM users to share with and the type of access (read/write or read-only) to grant them. Any IAM users who have been granted access will be able to see your development environment listed under the ‘Shared with you’ menu item in Services -> Cloud9 of their AWS Console.

Image 11 shows that the Cloud9 environment has been shared with IAM user ‘bcutler’ with ReadWrite permissions. In addition to the ability to see real time updates from others as they type, a group chat window makes it easy to communicate with co-collaborators. For example, IAM user ‘bcutler’ is actively editing the print statement on line 37 and has posted in the group chat on the right side of the screen.

Image 11: Cloud9 environment shared with an IAM user

The above features are simple yet effective, making collaborative cloud development possible for teams of all sizes.


Testing and Deploying Lambda Functions in AWS Cloud9

While Cloud9 provides a clean, snappy editor for the development of Lambda functions, arguably its greatest strength is the ability to execute and debug Lambda functions locally. As you test, you can provide payloads to mock input events commonly used as triggers for functions. The snapshotting Lambda function was designed to run on a schedule, as defined by a CloudWatch Event Rule. The screenshot below (Image 12) shows how an input payload (green and red text) can be used when testing your function:

Image 12: Local execution of Lambda in Cloud9

Another particularly useful feature of the Cloud9 IDE is the ability to set breakpoints within your Lambda function, as shown to the right of Image 12. This allows you to check local variable values as you step through the execution. Finally, Cloud9 provides you with information on memory usage and execution time, helping you to allocate the correct resources and set timeout values before deploying it remotely to your AWS account.

Once you have tested your function thoroughly and are ready to deploy it to your AWS account, the process couldn’t be simpler. In the menu to the right of your developer environment, simply right click on your function listed under local functions and click Deploy. Within a matter of minutes, the latest version of your code will be pushed to your AWS account and visible within Services → Lambda.


Summary

Although in its infancy, AWS Cloud9 is a welcome and long overdue addition to the AWS suite of services. Since its inception, the most common complaint about Lambda has concerned difficulties in debugging functions. However, the simple yet effective tools provided by Cloud9 are a clear step in the right direction. Coupled with a streamlined editor and cross collaboration functionality, AWS Cloud9 provides an attractive option for developing your next Lambda function. I’ve only scratched the surface in terms of available functionality, so I urge you to check it out for yourselves. With a few clicks, you too could be developing on Cloud9 in no time!