Installing and Configuring the AWS CLI

You’ll need to use the AWS CLI if you want to create or interact with an EMR Cluster using commands. Take these important steps to install and configure it.

The AWS Command Line Interface (AWS CLI) is a command-line tool that allows you to interact with AWS services using commands in your terminal/command prompt.

AWS CLI enables you to run commands to provision, configure, list, delete resources in the AWS cloud. Before you run any of the aws commands, you need to follow three steps:

1. Install AWS CLI
2. Create an IAM user with Administrator permissions
3. Configure the AWS CLI

Step 1. Install AWS CLI v2

Refer to the official AWS instructions to install/update AWS CLI (version 2) based on your underlying OS. You can verify the installation using the following command in your terminal (macOS)/cmd (Windows).

See the sample output below. Note that the exact version of AWS CLI and Python may vary in your system.

Mac/Linux/Windows: Verify the successful installation of AWS CLI 2

Step 2. Create an IAM user (if you are using your own AWS account)

In this step, you will create an IAM user with Administrator permissions who is allowed to perform any action in your AWS account, only through CLI. After creating such an IAM user, we will use its Access key (long-term credentials) to configure the AWS CLI locally.

Let’s create an AWS IAM user, and copy its Access key.

AWS Identity and Access Management (IAM) service allows you to authorize users / applications (such as AWS CLI) to access AWS resources.

The Access key is a combination of an Access Key ID and a Secret Access Key. Let’s see the steps to create an IAM user, and generate its Access key.

• Navigate to the IAM Dashboard, and create an IAM user.

Add a new IAM user

• Set the user details, such as the name, and access type as Programmatic access only.

Set the user name, and type (mode) of access

• Set the permissions to the new user by attaching the AWS Managed AdministratorAccess policy from the list of existing policies.

Attach the AdministratorAccess policy from the list of pre-created policies

• Provide tags [optional], review the details of the new user, and finally create the new user.
• After a user is created successfully, download the access key file (.csv) containing the Access Key ID and a Secret Access Key. You can even copy the keys and stay on the same page. Don’t skip this step as this will be your only opportunity to download the secret access key file.

Copy the Access key of the new user OR download the .csv file containing the Access key

Step 3. Configure the AWS CLI

You will need to configure the following four items on your local machine before you can interact with any of the AWS services:

1. Access key — It is a combination of an Access Key ID and a Secret Access Key. Together, they are referred to as Access key. You can generate an Access key from the AWS IAM service, and specify the level of permissions (authorization) with the help of IAM Roles. If you are using the Udacity-provided AWS Gateway and account, these credentials are provided in the popup that appears when you click “Launch AWS Gateway” in the course menu to the left.
2. Default AWS Region — It specifies the AWS Region where you want to send your requests by default.
3. Default output format — It specifies how the results are formatted. It can either be a json, yaml, text, or a table.
4. Profile — A collection of settings is called a profile. The default profile name is default, however, you can create a new profile using the aws configure --profile new_name command. A sample command is given below.
5. Session Token — If you are using the Udacity-provided AWS Gateway and Account, you will also need to add the session token from the AWS Gateway credential popup, as well as the Access Key and SecretKey from the popup as described here.

• Set the default profile credentials

Moving forward, you can use --profile <profile-name> option with any AWS command. This will resolve the conflict if you have multiple profiles set up locally.

The command above will store the access key in a default file ~/.aws/credentials and store the profile in the ~/.aws/config file.

Let the system know that your sensitive information is residing in the .aws folder

Mac/Linux: A successful configuration

Setting the Session Token

After a successful credential set-up, your “credentials” file will look like this one below. If you are using the Udacity-provided AWS account, you should have a session token name/value pair in your configuration as well.

Mac/Linux: View the credentials file using cat ~/.aws/credentials command

Step 4. Run your first AWS CLI command

• Check the successful configuration of the AWS CLI, by running either of the following AWS command:

The output will display the details of the recently created user:

Troubleshoot

If you are facing issues while following the commands above, refer to the detailed instructions here -

1. Configuration basics
2. Configuration and credential file settings
3. Environment variables to configure the AWS CLI
4. Using the Session Token

Updating the specific variable in the configuration

In the future, you can set a single value, by using the command, such as:

# Syntax
# aws configure set <varname> <value> [--profile profile-name]
 aws configure set default.region us-east-2

It will update only the region variable in the existing default profile.