Amazon Web Services (AWS EC2)
AWS Fundamentals & AWS EC2 Instances
Here, we will discuss the basic and important fundamentals of AWS. This tutorial will help you with the knowledge you need as AWS EC2 solutions architect and for interview preparation.
AWS Regions — These are the data centers across the world. It’s a geographical region. A region is a set of data centers or cluster of data centers. Each region has its own name & region code. For example, US-West(N. California) — us-west-1
What are the regional services? Few AWS services are region scoped, which means, those services are available in that region only. If you create an instance of a service in one region, you can access that service from anywhere around the world.
Availability Zones(AZs) — We call each group of logical data centers an Availability Zone. Each AWS Region consists of multiple, isolated, and physically separate AZ’s within a geographic area.
To know more visit AWS Global Infrastructure(click here) and which services are offered in which region? Check it out
IAM(Identity & Access management) — AWS Security for users, group & roles. You can secure each of your service in AWS using IAM. IAM is a global service and not a regional service.
Group — It’s a group of users. And you can set permissions to the group and users will inherit the permission.
Roles — Roles are not for users. Roles are specified for machines.
Policies — These are basically JSON documents which state what a group or role can do, that means, the permission for a group or role.
IAM Federation — Enterprises uses this IAM. Using this a user can log in using company credentials. Internally it uses SAML standards.
AWS EC2
AWS EC2 is capable of the below features:
- Launching virtual machines(EC2)
- Storing data on virtual drives(EBS)
- Distributing load across machines, i.e Load Balancing (ELB)
- Scaling services using autoscaling group(ASG)
Launching a virtual machine (EC2) — AMI(Amazon Machine Image) — This is the operating system which we want in our virtual machine. For example, Amazon Linux 2, Windows, etc.
Amazon Virtual Private Cloud(VPC) — Amazon Virtual Private Cloud is a commercial cloud computing service that provides users a virtual private cloud.
Subnet — It is basically a subnetwork within a region, which means an availability zone in a region where you want to launch your instance. For example — us-west-3a, where us-west-3 is the region and ‘a’ specifies an AZ. There will be some other AZs in that region with names like us-west-3b, us-west-3c, etc. Each region may contain a minimum of 2 AZs(a-b) and a maximum of 6 AZs(a-f).
Adding Storage — You need to add storage to your virtual machine also called EBS volume to install your operating system.
Security Group — Firewall for our EC2 instance.
Key-Pair — You need to use/create a key pair file (.pem file) with which you will be able to access(SSH) the EC2 virtual machine instance from your local machine.
Commands to access EC2 in a Linux machine —
ssh -i keypairfilename.pem ec2-user@(ec2-public-IP)
You will get an error after executing the above command. The private key pair file which you download is usually an unprotected file with permission 0644, which means the file is too open to get leaked. So, before executing the above command you need to change the permission for your keypair file as below:
chmod 0400 keypairfilename.pem
Note: You can also use Amazon EC2 Instance Connect provided in the AWS portal to access your machine.
Security Group
The security groups are network security in AWS. They control access to ports & how the traffic is allowed in(described by inbound traffic rule) and out(described by outbound traffic rule) of EC2 virtual machines. Security groups can be attached to multiple instances. Security groups created can be used only for a region. Those groups cannot be used for instances in a different region.
Note: If you do not have permission to access the EC2 instance, you will be timed out while accessing EC2. Remember, the security groups act as a firewall to the EC2 instance, which means EC2 instances are unaware of the unauthorized users trying to access it. Firewall blocks those unauthorized requests before reaching EC2. You can restrict the security group role just to an IP and access to a port. By default, all inbound traffics is blocked and all outbound traffics is authorized.
Note: If you are getting a ‘connection refused’ error instead of timeout while accessing EC2, then it might be an issue with the EC2 instance.
Elastic IP — When you stop and start an EC2 instance, its public IP may change(private IP won’t change). So if you want your EC2 instance restrict to same IP always, you have to use elastic IP. It’s a public IPv4. So, when you create an elastic IP, you own that public IP in global network. And one obvious thing is you can attach an elastic IP to one EC2 instance at a time. So, the advantage of elastic IP is that you can change your EC2 instance to a new instance with same public IP without breaking your application call to EC2. You can create 5 elastic IPs for your account by default.
Create Elastic IP → ‘Associate Elastic IP address’ to your EC2 instance
To disassociate elastic IP→EC2 Instance → Networking →Disassociate elastic IP address
Important !! If you are not using the elastic IP address created for your account, you will get charged for the IP. So you must release the elastic IP address if you are not using it.
Note: Instead of elastic IP the more common way is to use random public IP with a DNS name or using load balancers without any usage of public IP/DNS(discussed later).
Command to update your EC2 with all the packages — yum update -y
Install HTTP server — yum install -y httpd.x86_64
To start HTTP server — systemctl start httpd.service
To make it enable across reboots — systemctl enable httpd.service
Now, to launch the web page from your machine you have to add an HTTP security group for the server with port 80.
http://(public-IP):80
EC2 User Data — It is basically a script which we gets executed when our EC2 machine starts for the first time only when you create. So you can write script to update packages, install new softwares, downloading files etc.
An example script:
#!/bin/bash
yum update -y
yum install -y httpd.x86_64
systemctl start httpd.service
systemctl enable httpd.serviceEC2 Launch Types
On demand instances — (Short workload) — $0.10/hour
Pay according to usage time(billing/sec, after the first minute). So, mostly you get high cost. This is being used when you want any uninterrupted short workload. (No upfront payment)
Reserved instances — (1 or 3 years) — Known amount of time(Long workload)
1. Reserved — (Pay upfront) — 75% discount compared to on demand instance— Recommended for steady usage for your application for long term.
2. Convertible reserved instances (54% discount)— You can change the EC2 instance type.
3. Scheduled reserved instances(5–10% discount)(need for few weekdays specific time for running jobs etc.) — Required fraction of day/week or month.
Spot Instances — (Short workload) — 90% discount — You can lose these instances if your ‘max price’ goes below the current spot price. So, these are not recommended for database usage. It is used for workloads which are resilient to failure. So, when the current spot price goes over your max defined price, you lose the instance as they gets terminated.
Spot Block — Block spot instance for a specified time frame for few hours(1–6 hours) without interruption.
Spot Instance request — Requests a Spot Instance. The request provides the maximum price per hour that you are willing to pay for a Spot Instance. If you don’t specify a maximum price, the default maximum price is the On-Demand price. When the maximum price per hour for your request exceeds the Spot price, Amazon EC2 fulfills your request if capacity is available. A Spot Instance request is either one-time or persistent. Amazon EC2 automatically resubmits a persistent Spot Instance request after the Spot Instance associated with the request is terminated. Your Spot Instance request can optionally specify a duration for the Spot Instances.
A Spot Instance pool is a set of unused EC2 instances with the same instance type (for example, m5.large), operating system, Availability Zone, and network platform. When you make a Spot Fleet request, you can include multiple launch specifications, that vary by instance type, AMI, Availability Zone, or subnet. The Spot Fleet selects the Spot Instance pools that are used to fulfill the request, based on the launch specifications included in your Spot Fleet request, and the configuration of the Spot Fleet request. The Spot Instances come from the selected pools.
Spot Fleet —A Spot Fleet is a collection, or fleet, of Spot Instances, and optionally On-Demand Instances.
The Spot Fleet attempts to launch the number of Spot Instances and On-Demand Instances to meet the target capacity that you specified in the Spot Fleet request. The request for Spot Instances is fulfilled if there is available capacity and the maximum price you specified in the request exceeds the current Spot price. The Spot Fleet also attempts to maintain its target capacity fleet if your Spot Instances are interrupted.
You can also set a maximum amount per hour that you’re willing to pay for your fleet, and Spot Fleet launches instances until it reaches the maximum amount. When the maximum amount you’re willing to pay is reached, the fleet stops launching instances even if it hasn’t met the target capacity. It optimizes your cost with lowest price available for the instances.
Dedicated Instances (75% discount — 1 year)—Dedicated Instances are Amazon EC2 instances that run in a VPC on hardware that’s dedicated to a single customer. Your Dedicated instances are physically isolated at the host hardware level from instances that belong to other AWS accounts. Dedicated instances may share hardware with other instances from the same AWS account that are not Dedicated instances.
Dedicated Host — (3 years — on demand price) —(More expensive) — Book entire physical EC2 server & you can control instance placements(Dedicated Hosts give you additional visibility and control over how instances are placed on a physical server). They gives you visibility of underlying sockets/physical cores of the hardware. Useful for the software with licensing model.
EC2 Instance Types — https://ec2instances.info/
R — RAM oriented — in memory caches applications
C — CPU Oriented — applications with high computing capacity
M — Balanced with all types — web applications
I — I/O oriented — databases
G — GPU oriented — video rendering applications/machine learning
Burstable Instances — T2/T3
When the machine needs suddenly high computation power, it utilizes the CPU heavily called burst. If the machine burst, then it utilizes burst credits. Again when it goes to normal state, burst credits gets accumulated again. If your application burst the machine always, then you can face performance issues.
T2/T3 Unlimited burst — It gives unlimited burst credit balance. You have to pay extra money if you go over burst credit, but you will never lose performance.
EC2 AMIs — AMIs are used to create our instances. AWS EC2s comes with lot of images like Linux, Windows etc. You can create your own custom image. Custom AMIs which you built are for specific AWS region, but you can copy it to other region. We can use/rent other public AMIs. By default AMIs are private and locked for your account only. To change you have to use modify image permissions. AMI uses Amazon S3 as storage.
Now using that image, you can launch a new EC2 instance like shown below:
EC2 Placement groups —
Cluster —All your EC2s are in same AZs, same hardware. One can use when one need high network speed/bandwidth with extremely low latency. But we have a high failure risks if the hard fails.
Spreads — All your EC2 instances are distributes across different hardware in a region. One can use this to increase maximum availability of the application. But we have a limitation here like we can have max 7 instances per AZ per placement spread group.
Partition — Partitions are set of racks in an AZ. We can create max 7 partition in an AZ and max 100s instances.
Elastic Network Interfaces (ENI)— They represent a virtual network card. They give EC2 instance access to the network. ENI can have a primary private IP(IPv4) and one or more secondary private IPs(IPv4), one public IP, one or more security groups. You can create ENI independently and attach or move them anytime to EC2 instances to save them from failover. They are bounded to an AZ.
EC2 Hibernate — When you create an instance, the instance gets created with EBS volume(storage) where your OS gets installed. When you stop the instance, then data on the EBS is intact for the next start. When an instance terminates, the data on any instance store volumes associated with that instance is deleted. By default, Amazon EBS root device volumes are automatically deleted when the instance terminates. However, by default, any additional EBS volumes that you attach at launch, or any EBS volumes that you attach to an existing instance persist even after the instance terminates. So, when again you start your instance after you have stopped, it takes long time to start if your instance or application takes a long time to bootstrap.
You can use hibernation to pre-warm the instance. To pre-warm the instance, you launch it with hibernation enabled then bring it to a desired state and hibernate it, ready to be resumed to the same state as needed.
When you hibernate an instance, Amazon EC2 signals the operating system to perform hibernation (suspend-to-disk). Hibernation saves the contents from the instance memory (RAM) to your Amazon Elastic Block Store (Amazon EBS) root volume. Amazon EC2 persists the instance’s EBS root volume and any attached EBS data volumes. When you start your instance:
- The EBS root volume is restored to its previous state
- The RAM contents are reloaded
- The processes that were previously running on the instance are resumed
- Previously attached data volumes are reattached and the instance retains its instance ID
That’s all for AWS EC2 instances. Hope you enjoyed :)
