How to Install and Configure awscli on Windows or Mac, then confirm it worked by pulling a file from S3

So, you’ve been given an account, and access keys. Now what?

This blog will go over how to configure your AWS profile on both Windows and Mac machines so that you can successfully pull files from S3 to your local in a Jupyter notebook.

Prerequisites:

Python — pip, in particular

Step One: Install awscli

https://docs.aws.amazon.com/cli/latest/userguide/install-macos.html#awscli-install-osx-pip

pip install awscli

Then Update your environment variables:

Windows:

To find where awscli is installed, type the following into your command line:

WHERE aws

We’re going to add the filepath leading to aws.exe to our environment variables so your command line can recognize it.

Navigate to your Environment Variables (You can do this by typing “Environment Variables” in your taskbar search)

Mac:

Type the following into your cmd after installing

source .bashrc

Step Two: Configure awscli

Configuring the AWS CLI - AWS Command Line Interface

Since this is your work profile and you’ll likely also have personal and school profiles, configuring for multiple profiles is best.

aws configure --profile user2 
AWS Access Key ID [None]: AKIAI44QH8DHBEXAMPLE 
AWS Secret Access Key [None]: je7MtGbClwBF2Zp9Utkh3yCo8nvbEXAMPLEKEY 
Default region name [None]: us-east-1 
Default output format [None]: json

To check that this worked, you can navigate to file `~/.aws/credentials` and see that your keys and profiles are stored there.

When you need to switch profiles, type in cmd, make sure there are no spaces around the equals sign.

export AWS_PROFILE=name_of_aws_profile

Or use environment variables.

Step 3: Confirm everything is working properly by trying to load in data from S3

Remember these keys are for your eyes only! Treat them as passwords. If they happen to be made public anywhere on the internet, bots will find them and try to use your EC2 resources. Both GitHub and AWS have steps in place to look out for this, including e-mail notifications and suspending your AWS account if they suspect your account has been accessed by someone besides you. You can recover your account by following all steps AWS Support provides.

Since these keys are so sensitive, we will be using a way to reference them without ever having to type them out or store them in a file: boto3.Session().get_credentials()!

This is a sister-blog to my entry about Thomas Edison State University’s (TESU) open source materials accessibility initiative. Medium post / Github repository

The initial blog I composed for this project was specific and borders on the line of unwieldy, so this blog is the first in a series I will be writing to take deep dives into separate aspects of the TESU project and make the material more accessible.