ICONSENSUS Resiliency (Part 2: Sys Ops Security)
In Part 1 of this series, we discussed the need for resilient nodes as a key to a successful decentralized ICON network. Part 1 covered several aspects of resiliency at a high level. In this iteration, we will take a deeper look into System Operations (Sys Ops) and go over some necessary security practices and options as it relates to setting up a Node on AWS. This is certainly not a comprehensive list of best security practices, but it offers some helpful suggestions to start with and we hope it will fuel a larger effort to improve security across all of the P-Rep teams.
Securing an AWS account:
Your AWS account is very important to secure. Since root access to the AWS account can then be used to modify or remove any ICON Nodes, as well as create new instances and rack up bills, it is vital to ensure this account stays safe. This section assumes you have recently created an AWS account. For this tutorial, I created a dummy AWS account and launched a few AWS services. The images should be similar to what any new AWS account would see (but are applicable to any account).
AWS offers a service called Trusted Advisor, which checks the configuration and setup of multiple AWS areas to ensure best practices. These areas are cost optimization, performance, security, fault tolerance, and service limits. We will focus on security. Trusted Advisor can be accessed via clicking ‘Services’ and then searching ‘Trusted Advisor’ in the search box. Click on the ‘Trusted Advisor’ result, click the refresh icon in the upper-right corner, and then wait for the page to refresh. The page will show a list of recommended actions with check marks next to actions that have been completed, a triangle icon for actions that should be investigated, and a red circle for recommended actions. We will explore one recommended action as it relates to Sys Ops.
Click the drop down on the ‘MFA on Root Account’ to see more details, along with a link with a ‘how to guide’ that provides step-by-step instructions for the action. We will go ahead and do this.
Now if you run Trusted Advisor again, the ‘MFA on Root Account’ alert has changed to completed. The Trusted Advisor free support contains important high-level actions, but is limited. For more detailed checks and higher levels of protection, you can switch your account support plan. The Business Support plan is $100 / month and provides the full set of checks that AWS supports. Additionally, this plan provides enhanced technical support via 24/7 phone, email, and chat access to AWS Support Engineers. I think the Business Plan is definitely a support plan option that warrants consideration for serious P-Rep applicants who plan to use AWS. The developer support plan is much more limited and does not have 24/7 access. The basic (free) support plan does not have any personalized technical support.
Securing the root account is necessary, but we should not be using the root account if we don’t have to. It is best to secure root and then lock away the key. It is never a good practice to use the root account of any system for day-to-day tasks. There are two reasons for this:
1. As root, one has the privileges on the system to do anything and cause serious issues if not careful (whether through commands that are not fully understood or even something simple such as a mistyped command).
2. The more often you access an account, the more opportunities there are for hackers to take over that account (some examples are through social engineering attacks, man in the middle attacks, packet sniffing, etc).
The ‘sudo’ command exists in Linux in order to accomplish administrative tasks without loggin into the system as root. The AWS root account has permissions to all AWS resources, billing, contact information, and has no limits within AWS. Hackers go for the highest level of privileges they can, and obtaining root privileges is the top of the mountain in terms of hacking. From there, your system is pwned and hackers have unlimited access to do whatever they want / need.
We will go ahead and setup users as needed, so we do not have to login as the root account. AWS uses the Identity and Access Management (IAM) for this purpose. To successfully operate an ICON Node, an administrator is still needed. Thus, we will go ahead and add one. Navigating to the IAM Service, click on ‘Users’, and click ‘Add User.’ Then enter the user name. For this, we will just use a simple ‘Administrator’ name. Then there is the option to autogenerate a password and e-mail the user, allowing the user to login and prompting them to change the password at the next login. Or one can generate their own password at this time (this would be the option to choose if you are the administrator).
After the user is added, the next step is to set permissions for this user. Permissions can be set in three ways: by using group permissions, copying another user’s permissions, or using pre-made permissions policies from AWS. Since AWS already has an administrator permissions policy, we will use this. This is the permissions that your Sys Ops Administrator should have.
It is important to note that P-Rep teams will likely have many users with varying permissions. For these instances, the team admin should create IAM groups and assign permissions. The permissions can be assigned based on functional, organizational, geographical, or even project settings. For example, there may be a few users who work on the EC2 Linux Instance and need to ensure it is constantly up and running and properly configured. These users could have a custom policy that would give them full permissions for EC2 instances (including the option to create new instance). One can also assign permissions for users to only have the ability to access existing resources and not create new ones (as that could create new billing concerns). One can also assign temporary credentials that expire within a certain timeframe. This is useful for contract work, to further limit access and not have to manually revoke credentials after the project timeline is up. The opportunities are endless and provide maximum flexibility and security options. In general, it is best practice to ensure users have the least privileges necessary to accomplish their given tasks. One can always add more privileges as needed, but someone with privileges that they don’t need can cause irrevocable and unnecessary harm. In the case of the administrator, one should ensure they have enabled 2FA as well. While this account does have some account limitations, it is still very powerful and has many of the same privileges as the Root account, and should be handled with care.
Next, we will create a group for access to the EC2 instances. Navigate to the IAM service and click on ‘Groups’ and then ‘Create Group.’ We are first asked to name the group. We will create a group named ‘NodeDev.’ We then assign policies to the group. We will assign EC2 Read Only Access and also Container Service for EC2 Role, as that looks helpful for running a node. We selected Read Only Access because we want the node developers to be able to monitor the EC2 nodes, but not create new ones or change them. We will create an account for NodeDev users to log into the EC2 instance for management of the node via SSH, and thus they can do their job without having full EC2 AWS permissions. We will leave those permissions to the Sys Ops Admin.
We continue and create the group. Let’s now examine the group we just created. Go to the groups and click on ‘NodeDev.’ Next, click permissions, and then click ‘show policy’ for the EC2 Container Service.
Upon inspection, we realize we actually don’t want the NodeDev to have all of these permissions because they aren’t necessary for our current ICON Node EC2 instance configuration. If you’re unsure, you can click ‘simulate’ policy and you will be logged in to AWS with those policy permissions and can determine if you set the permissions exactly how you wanted to. In this case, we can easily remove this policy by clicking ‘Detach Policy.’ Then the resulting policy is just the EC2 Read Only Access, and we are done.
Since users will be creating their own passwords, we want to ensure there are proper minimum requirements for the password, to help prevent their accounts from being hacked. While it’s impossible to ensure complete account security, this will certainly help. The password policy can be found at the IAM service and then ‘Account Settings.’
For a quick review of user status, ‘Credential Report’ can be selected and then a report can be downloaded, listing all users and the status of their credentials. Obviously this will not list their passwords, as AWS does not store that information, however, it will list their last login, when they last changed their password, and other useful information.
This article provided an introduction to Sys Ops Security with AWS and provided some best practices for securing the AWS root account, setting up an administrative user, and showed how to setup groups and users with different permissions. For those who are using AWS, we hope this helps get you started on securing your account and setting the proper permissions. We would be happy to answer any questions you may have and also would love to hear suggestions you have for further securing our accounts. Please stay tuned for Part 3 of this series, as we dive into hardening our Linux EC2 Instance.