Case Study: How I managed to restore an AWS root account against (literally) all odds

Intro

In my most recent role as a DevOps Tech Lead, I managed AWS accounts for our clients.

The reason is that some of them lacked the technical knowledge of how to handle an AWS account and they pretty much wanted us to take care of the security concerns on their behalf. While keeping ownership of the root account.

This is how things are handled for most of our clients. We are creating the account, creating our own IAM users, handling the password and MFA to the client, and that’s it.

Let me tell you a story about one of those clients, and what we have learned from it.

If you get it you get it

The Story

I created this account couple of years ago and ran through all the root account best practices, which are:

• Delete the keys (I understand that today the root account is created without any keys, which is nice).
• Attach an MFA device (one of the client’s cell phones).
• Create an Administrators group.
• Attach the Admin team members’ user accounts to the group.
• Log out.
• If you want to feel good with yourselves you can also create a CloudTrail rule to be alerted when someone is accessing the root account for any reason.

A few years passed by, no action has been made with the root account and everything was fine.

Somewhen last year, my company’s management wanted to make sure that root accounts for all clients are accessible. This is a very good approach. When one of your Admin accounts will be compromised, the only way to gain access is by using the root account, so it‘s a really good practice to make sure you can log in when it’s needed.

To gain access to the root account, I needed the root account password and MFA code, which are all being held by the client. But unfortunately, the client didn’t have any idea what is the password, nor where he can find the MFA code.

Creating a new password was quite easy. I ran the “forget my password” procedure, an email was sent to the clients’ company email address and we created a new secured password.

But what about the MFA?

To bypass MFA to get to a root account, you need only 2 things:

1. Access to the root account’s email address.
2. Receive a phone call to your contact’s Phone number (can be found under the “Contact Information” section to any IAM user with the relevant privileges).

We already know we have access to the relevant person to get the verification email. all we need is a person to answer an automated call from AWS and press a six-digit code which I can send him. right?

Then no. The client has no idea who this number belongs to. And all I think about is the poor IT guy that must have left years ago and now receiving weird phone calls in the middle of the night. No kidding, I tried to call like 20 times using this process. Maybe someone left the phone under the sofa cushions or something.

So let’s summarize for now. we want to access the AWS root account without:

• An MFA device.
• An Access to the contact phone number.

The only thing we have is:

• Root email access.

What did we do?

Do you remember my list of best practices when creating an AWS account? so let me take you back to one of the statements:

Create an administrator group.

Attach the Admin team members’ user accounts to the group.

Right before I was about to give up I reminded myself that I can control exactly what I need. I can change the contact’s phone number.

At this point, if you don’t have access to a user who is part of the Admin group, or has “aws-portal:ViewAccount” and “aws-portal:ModifyAccount” permissions, you are pretty much… you know.

But if you are working with at least one Administrator who is an IAM user (as you should) you can change the phone number.

Using my IAM administrator user I changed the root account’s phone number to my phone number (poor ex-IT guy, now I know how it feels), and ran the MFA bypass process.

That’s it. I’m in.

Of course, I don’t just sit back and rest on my laurels. I immediately remove the unused MFA device and attach a new one using my phone.

Now when we have access to the client’s root account, we can calmly reach them and asked them for another phone number and a new device to hold the MFA. until next time.

What do we learn?

Now let’s be real for a second and understand why this method is working for us now, but it will not work when you will face a breach in your account.

• You usually don’t need your AWS root account. meaning that when you do, you will probably need it fast. and this process I was just describing is not fast at all. It’s cumbersome and requires many people to work together. In our case, people from different companies and different time zones.
• If somebody gains access to one of your administrative IAM accounts, the first thing he will do (in case he’s smart) is to revoke all other accounts’ permissions. meaning you will not be able to change your phone number to gain access.

This means that you HAVE to know that you are always able to log in to your root account immediately when there’s a situation that requires this.

Also, it will not hurt to have a quarterly check to make sure you can access the root account. and if you find out that you can’t, well, now you know what to do.