Oct 12, 2022

7 min read

Automated Creation of an SSH Key for an AWS User

ACM.78 Automated creation of an AWS EC2 SSH key stored in AWS Parameter Store

This is a continuation of my series on Automating Cybersecurity Metrics.

I wrote about different authentication mechanisms in the last post.

Mechanisms of Authenticating to a Linux VM (EC2 Instance) on AWS

ACM.77 Considerations for how you access virtual machines in a cloud environment

medium.com

Now let’s look at how we might automate the deployment of an SSH key that allows a user to log into an EC2 instance.

This implementation has some security problems that we will fix in the next few posts. I thought it was going to be simple...but it wasn't.

An SSH key identifies a user (as do any credentials you assign to a user)

An SSH key should belong to a single user. It identifies that user as the person who connected on a certain day and time to a host. We can going to add our SSH key creation code to our common IAM functions and have the IAM team manage the deployment of SSH keys — but the IAM team should never have access to the keys that belong to other users. See the last post for a description of non-repudiation and why it is important for security.

We are going to create a key associated with a user at the time we create the user. The key is going to have the same name as the user. We’ll create it in the IAM directory of our code using the IAM CLI profile as it is tied to user identity.

Create an SSH Key and store it in AWS Parameter Store

Here’s some code that creates a key and stores it in AWS Parameter store. I put in in my user_functions.sh file in my IAM Users directory. The profile with reference the IAM Profile included in the file.

I’m going to add a parameter to my create user function that indicates whether a key should be created for that user or not.

I’m going to check if that value is “y” and if so, call the function to create the user key.

Over in my deploy.sh file I’m going to pass in “y” or “n” indicating whether I want to create a key or not for each user. For now, I am only going to create an SSH key for the Developer user.

OK now run the deploy script.

Error: No permission. What permissions do we need?

ec2 create-key-pair
ec2 wait key-pair-exists
aws ssm put-parameter

What is interesting to me is that an SSH key is really an IAM control, but it exists under EC2. That has implications for your IAM permission management. If you give someone ec2.* permission that person can do two things which are super critical for cloud security:

Change networking
Deploy SSH keys

So a person could set up a new network with access to your sensitive resources in whatever account they are in and install a new ec2 instance with an SSH key, login, and connect to other resources in your network. If anything is not protected with IAM controls, encryption, and key policies, then that user might be able to get to it.

The moral of the story: Be careful with * in IAM Policies!

We still need to go back and limit our network admin permissions to a zero trust policy when we are through deploying network controls. For now let’s add the necessary permissions to our IAM Admin group role policy.

Use the deploy_iam_role.sh deployment script to deploy the role. Deploy again. My script worked the first time but not the second time because I need to add a line to delete the key if it already exists and then add that permission to the IAM admin group policy.

You may have noticed I’m using a command with the word “Waiter” in it. That keyword causes the command to wait for the exists of a resource, in this case a key pair.

Apparently the KeyPairExists command also requires ec2.DescribeKeyPairs.

And…it works.

It works! We’re done right?

Not quite. Let’s ponder our script for a minute. How does it work? It downloads the SSH key and stores the output in a file. It then manipulates that file output to pull out a valid.pem file we can use as an SSH key to log into an AWS host. (It has stored the public key on AWS for use with EC2 instances.)

So where did those two files end up?

They are in our home directory (specified by ~ below):

Anyone who can access this host — or it’s backups — would have access to those keys. Someone could also detach the EBS volume (virtual drive) and attach it to another instance. Someone could create an image of this machine and launch another one just like it to access those files.

Well, we could delete those keys. That should solve the problem right? Not exactly. I’ll show you why in the next post.

What other problem do we have? We had to give the IAM admin full access to create and delete parameters in SSM. Is that OK? Maybe it is, maybe it isn’t. It depends on whether you store sensitive data in Parameter Store or not and whether you encrypt it.

Speaking of encryption, we didn’t encrypt our credentials. Anyone who has access to all of Parameter store can see them. As already mentioned, if we use Parameter store we can’t assign a resource policy to restrict access to our parameters. There’s no way to create a parameter store administrator who can restrict access to parameters.

We could use an encryption key with to solve this problem if we want to use a resource policy in addition to an IAM policy. If we create a KMS key and assign it to a user then we can create the credentials and even the IAM administrator can’t see them as only that user would be allowed to encrypt and decrypt with that key. What’s the potential downside of that approach? We have to pay $1 per encryption key on AWS last time I checked. If you have 11,000 developers like Capital One did when I was there, that’s $11,000 per month. Is that ok? Maybe it is, maybe it isn’t.

Is there an alternative? We can store the credentials in Secrets Manager. If we store the credentials in Secrets manager, then we can create a policy to auto-rotate the credentials. We can also create a policy that only allows the specific developer who owns the credentials to access them. We could use a generic key for encrypting credentials, but then we’re still paying more for Secrets Manager. What’s the difference?

Overall, if I’m looking at this correctly, parameter store with a KMS key per user would cost $1.05 per month whereas Secrets manager with a single KMS key for all developers but a policy restricting each developer to only access their own SSH Key secret would cost .40 per user plus $1 per month. If you only have one user then it’s $1.05 vs. $1.40. If you have two users, then Secrets Manager is looking like a better option. The operations cost the same.

Alright we need to think about how we’re going to fix the above issues. Stay tuned for the next post.

Teri Radichel

If you liked this story please clap and follow:

Medium: Teri Radichel or Email List: Teri Radichel
Twitter: @teriradichel or @2ndSightLab
Requests services via LinkedIn: Teri Radichel or IANS Research