Using Boto3 to Automate the stoppage of Ec2 Instances to save money

Referenced resources: Using Python boto3 library & other AWS Services
https://aws.amazon.com/sdk-for-python/

https://boto3.amazonaws.com/v1/documentation/api/latest/guide/ec2-example-managing-instances.html

https://boto3.amazonaws.com/v1/documentation/api/latest/index.html

Objectives:

Project A

Scenario: Our DevOps engineering team often uses a development lab to test releases of our application. The Managers are complaining about the rising cost of our development lab and need to save money by stopping our (for this example) 3 ec2 instances after all engineers are clocked out.

Create a Python script that you can run that will stop all instances. Push your code to GitHub and include the link in your write up.

ADVANCED

We want to ensure that only our development instances are stopped to make sure nothing in Production is accidentally stopped. Add logic to your script that only stops running instances that have the Environment: Dev tag.

COMPLEX

Use your script in a Lambda function using Python 3.7 or higher runtime. Make sure the lambda runs on a set schedule daily. No one should be working in the Dev environment past 7pm. (Note: to test you may need to modify the time accordingly, so you aren’t waiting for 7pm)

Prerequisites:

· AWS account

· IDE tool

· Familiarity with Python and Boto3

· IAM user account with administrative rights

· AWS CLI — understanding on how to use and install

Key notes:

1. Boto3 — Is an AWS SDK to give the leverage of creating, managing and configuring your AWS services using Python. Using this Python API allows you to advantage of building your application on top of using it to manage AWS infrastructure services.

2. SDK — A set of tools for third-party developers to use in producing applications using a particular framework or platform.

Let’s get started!

We will first navigate to our Cloud9 IDE to install both boto3 and aws cli.

pip install boto3

pip3 install awscli

To start our Python script, we will first require a new branch and file that will store our code.

We will be pushing and merging this local branch with our remote GitHub repository.

Instance creation for both Dev and Proud environments.

Head to the dashboard and we will get the AMI ID.

I will be using AWS CLI to create the Ec2 instances for both development and production environments.

# Create Prod Ec2 instances using CLI 
aws ec2 run-instances --image-id ami-0747e613a2a1ff483 --instance-type t2.micro --count 2 --tag-specifications 'ResourceType=instance,Tags=[{Key=webserver,Value=production}]'

# Create Dev Ec2 instances using CLI 
aws ec2 run-instances --image-id ami-0747e613a2a1ff483 --instance-type t2.micro --count 2 --tag-specifications 'ResourceType=instance,Tags=[{Key=webserver,Value=development}]'

Example of the instances created from CLI output.

Verify the deployment of the Ec2 instances via the AWS dashboard. Can see that there are 8 instances including our Ec2 instance that is running our Cloud9 IDE.

Now we will prepare our python script to stop all running instances.

Change current working directory to where the file is in.

We will first stop all our running instances first of course only excluding the instance our Cloud9 IDE is running on. We will be using the code below:

#!/usr/bin/env python3.7

#Import all boto3 packages
import boto3


#Assign the ec2 variable to it value of the boto3.client module 
ec2 = boto3.resource('ec2')
client = boto3.client('ec2')


# Initialize list of all instance ID's discluding the instance the Cloud9 IDE is running on
 
ec2 = ["i-0cdf332d8a011fdee", "i-080a2ef8881e57906", "i-02175a4e91b320d90", "i-037df89dd11dbc10c", "i-08362d37085d8354b", "i-00c5ba0fcda0a4ef9", "i-0098a69a7b0043007"]

#stop all instances using the list value that was passed to the ec2 variable
client.stop_instances(InstanceIds = ec2)

#Print the ec2 instances that have been put on a stopped state
print('The following EC2 Instances have been stopped: ' + str(ec2))

Checking our AWS EC2 dashboard we can see that all are set in a “stopped” state other than the instance our IDE is running on currently.

We will now hit the advanced portion of the project since we successfully stopped all running instances. We will comment out the code that will stop the instances and its printout and add code to start all the instances back up to complete the next objective.

Confirmed! They’re all back up and running including our Cloud9.

Now we will stop all running instance tagged Environment: Dev. Nothing in production should be stopped.

Run the following code:

#!/usr/bin/env python3.7

import boto3

#Assign the ec2 variable to it value of the boto3.client module 
ec2 = boto3.resource('ec2')
client = boto3.client('ec2')

ec2_running ={'Name':'instance-state-name', 'Values':['running']}
ec2_tag ={"Name":"tag:webserver","Values":['development']}

for instance in ec2.instances.filter(Filters=[ec2_running,ec2_tag]):
    instance.stop()
 
print('The following EC2 Instances have been stopped: ' + str(ec2))

Can see that there are a few in a stopped state now:

Checking out one of those same ones to check if it has the development tag value.

There you go we just saved the managers and CFO tons of Money by automating this process for the environment.

Check out my previous posts on how to push all our codes to our remote GitHub repositories.

Thank you for stopping by and I hope that you’ve learned a thing or two. Feel free to comment like and re-share as always feedback is welcomed.