Starting with AWS Boto3

6 min readMar 24, 2020

Quick guide to understand how to start working with AWS SDK

Introduction

First thing’s first, What is Boto3? The easiest thing for me would be to copy the official reference from Amazon about its SDK:

Boto is the Amazon Web Services (AWS) SDK for Python. It enables Python developers to create, configure, and manage AWS services, such as EC2 and S3. Boto provides an easy to use, object-oriented API, as well as low-level access to AWS services.

In my own and simple words… Boto3 consists of a set of Python functions specific to to interact with the Amazon Web Services.

Curious fact: According to his creator “Mitch Garnaat”, Boto was named after the fresh water dolphin native to the Amazon river. These dolphins swim navigate the forest ecosystems with ease, of which boto also allows us to navigate the amazon ecosystem with ease.

What You’ll Learn

In this Guide, you will learn the basics on the official AWS SDK library for Python, known as Boto3.

Installation
Configuration
Basic Concepts
Using boto3

Installation

Install the latest Boto 3 release via pip:

pip install boto3
# You may also install a specific version:
pip install boto3=1.0.0

Configuration

Before you can begin using Boto 3, you should set up authentication credentials. Credentials for your AWS account can be found in the IAM Console. You can create or use an existing user. Go to manage access keys and generate a new set of keys.

If you have the AWS CLI installed, then you can use it to configure your credentials file:

aws configure

Alternatively, you can create the credential file yourself. By default, its location is at ~/.aws/credentials:

[default]
aws_access_key_id = YOUR_ACCESS_KEY
aws_secret_access_key = YOUR_SECRET_KEY

You may also want to set a default region. This can be done in the configuration file. By default, its location is at ~/.aws/config:

[default]
region=us-east-1

Alternatively, you can pass a region_name when creating clients and resources.

This sets up credentials for the default profile as well as a default region to use when creating connections.

Basic Concepts

Before start using Boto, You need to learn a few concepts to understand how actually use it.

Boto 3 is built a top of a library called Botocore, which is shared by the AWS CLI. Botocore provides the low level clients, session, and credential & configuration data. Boto 3 builds on top of Botocore by providing its own session, resources and collections.

Modules are typically split into two categories, those which include a high-level object-oriented interface and those which include only a low-level interface which matches the underlying Amazon Web Services API (These interfaces are resources and clients respectively. We’ll take a look at both of them shortly).
Some modules are completely high-level (like Amazon S3 or EC2), some include high-level code on top of a low-level connection (like Amazon DynamoDB), and others are 100% low-level (like Amazon Elastic Transcoder).

Clients: Are low level service connections with the following main features:

Support all services.
The outputs are returned using Python dictionaries.
We have to traverse these dictionaries ourselves.
Automatic paging of responses (Paginators)
A way to block until a certain state has been reached (waiters)

Let’s take a look at the two points I’ve highlighted in bold. First, let’s use the EC2 Client to create an instance and illustrate them:

import boto3

aws_access_key_id = ''
aws_secret_access_key = ''
region_name = 'ap-southeast-2'

session = boto3.session.Session(aws_access_key_id=aws_access_key_id,
                                aws_secret_access_key=aws_secret_access_key,
                                region_name=region_name)

ec2client = session.client('ec2')

client_instance = ec2client.run_instances(
    ImageId='ami-30041c53',
    KeyName='Keys',
    MinCount=1,
    MaxCount=1,
    InstanceType='t2.micro')

Therun_instancegives us a large amount of useful information. For example:

Request Syntax: How we can customize our instance under the section.
Response Syntax: What output we can expect to see.
Return type: The format in which the output will be provided. (Recall that Clients will always have a return type of dict.

Let’s now check the contents of client_instance:

In [13]: client_instance
Out[13]:
{'Groups': [],
 'Instances': [{'AmiLaunchIndex': 0,
   'Architecture': 'x86_64',
   'BlockDeviceMappings': [],
   'ClientToken': '',
   'EbsOptimized': False,
   'Hypervisor': 'xen',
   'ImageId': 'ami-30041c53',
   'InstanceId': 'i-0e3a125b86073f341',# omitted

As we expect, it is a dictionary.

Along with these major features, Boto 3 also provides sessions and per-session credentials & configuration, as well as basic components like authentication, parameter & response handling, an event system for customizations and logic to retry failed requests.

Resources: a high level, object oriented interface

Resources represent an object-oriented interface to Amazon Web Services (AWS). They provide a higher-level abstraction than the raw, low-level calls made by service clients.”

In other words, Resources (where available), can do the same thing as Clients but produce outputs which are a lot easier to consume, once we get the hang of it.

The documentation then goes on to tell us that “These can conceptually be split up into identifiers, attributes, actions, references, sub-resources, and collections”. Let’s go ahead and look into what these are.

Identifiers, attributes, references, actions & collections

Where Clients are fairly easy to wrap your head around, there’s quite a lot more to Resources. Thankfully AWS’ documentation gives us a great starting point. The italicised points below is AWS’ take on things, while the non-italicised points are my comments:

Identifiers: Properties of a resource that are set upon instantation of the resource. When a resource is created, it is given an ID. This ID can then be used in subsequent Attribute and Action calls.
Attributes: Provide access to the properties of a resource. Attributes are lazy-loaded the first time one is accessed via the load() method. Used in conjunction with a resource’s ID, Attributes provide information about the specified resource. For example, passing an EC2 instance’s ID to the image_id attribute shows us the AMI that this instance is running:

In [61]: ec2resource.Instance('i-0afb49cac524f3191').image_id Out[61]:
'ami-30041c53'

References: Related resource instances that have a belongs-to relationship. While attributes require a resource’s ID, References are methods that belong to the Python instance’s object and therefore do not need the ID to be provided:

In 65]: resource_instance[0].vpc.id Out[65]: ‘vpc-9e22dcfb

Actions: Call operations on resources. They may automatically handle the passing in of arguments set from identifiers and some attributes.
Make something happen. For example, shut down an instance:

In [69]: resource_instance[0].stop() Out[69]: {'ResponseMetadata': {'HTTPHeaders': {'content-type': 'text/xml;charset=UTF-8',    'date': 'Thu, 31 Aug 2017 11:45:08 GMT',    'server': 'AmazonEC2',    'transfer-encoding': 'chunked',    'vary': 'Accept-Encoding'},   'HTTPStatusCode': 200,          
# omitted

Collections: Provide an interface to iterate over and manipulate groups of resources. Outputs which need to be iterated over, for example the IP addresses in a VPC or the number of volumes attached to an instance:

In for volume in resource_instance[0].volumes.all():
    print(volume.id)vol-0e6630fdcea489f65

Waiters: Provide an interface to wait for a resource to reach a specific state.

EC2 instance using a Resource

Let’s now go ahead and use a session to create an EC2 Resource to ilustrate them:

import boto3aws_access_key_id = ''
aws_secret_access_key = ''
region_name = 'ap-southeast-2'session = boto3.session.Session(aws_access_key_id=aws_access_key_id,
                                aws_secret_access_key=aws_secret_access_key,
                                region_name=region_name)ec2resource = session.resource('ec2')resource_instance = ec2resource.create_instances(
    ImageId='ami-30041c53',
    KeyName='Keys',
    MinCount=1,
    MaxCount=1,
    InstanceType='t2.micro')

Those with a keen eye for detail may have noticed that the configuration Client and Resource configuration is almost identical. The only differences being the following two bits:

session.resource as opposed to session.client
ec2resource.create_instances as opposed to ec2client.run_instances

However, that’s where the similarities end. For example, when we resource_instance we see a fraction of the information we saw when we did the same with client_instance :

In [11]: resource_instance
Out[11]: [ec2.Instance(id='i-0afb49cac524f3191')]

Wow, what a difference. The Client output a large amount of details about the instance whereas the Resource only shows us the instance’s ID. As a result of this, we’ll need to extract the ID to obtain more information on the instance itself.

To extract the this ID we use a similar technique to the one we used when extracting the ID from client_instance. Though while in that case we traverse a dictionary, this time we’re navigating an OOP instance:

In [12]: resource_instance[0].id
Out[12]: 'i-0afb49cac524f3191'

Using Boto3

To use Boto 3, you need to follow the next steps:

1.- Import it and tell it what service you are going to use:

import boto3

# Let's use Amazon S3 as resource
s3 = boto3.resource('s3')# Let's use Amazon AutoScaling as client
asg_client = boto3.client('autoscaling')

2.- Make a request and process response from the service:

# Print out bucket names
for bucket in s3.buckets.all():
    print(bucket.name)# Print out auto scaling groups names
asg_dict = asg_client.describe_auto_scaling_groups()
    for asg in asg_dict['AutoScalingGroups']:
        print (asg['AutoScalingGroupName'])

Congratulations!

If you’ve made it to the end of this post, you most certainly deserve a pat on the back. I didn’t intend for this to be such a length post, but once I got started I couldn’t stop :)

I hope you enjoyed it. Please feel free to drop me a message if you’d like me to expand on any part or parts of Boto 3 and I’ll be happy to oblige.