Mastering the Cloud: Module 6-Compute
Amazon Web Services offers many computing Services like:
1. ‘Amazon EC2’, provides resizable virtual machines.
2. ‘Amazon EC2 Auto Scaling’, supports application and availability to launch & terminate EC2 instances.
3. ‘Amazon ECR’, used to store and retrieve Docker images.
4. ‘Amazon ECS’, is a container Service that supports Docker.
5. ‘VMware Cloud’, enables us to manage a hybrid cloud with hardware
6. ‘Aws Elastic Beanstalk’ is, a simple way to run and manage web applications
7. ‘Aws Lambda’, Serverless Compute solution we pay only for the time we use the application.
8. ‘Aws EKS’, enables us to run managed Kubernetes on Aws.
9. ‘Amazon Light Sail’, provides a Service for building an application.
10. ‘Aws Batch’, provides a tool for running batch jobs at any scale
11. ‘Aws Fargate’, provides a way to run selected containers that reduce the need to manage Servers.
12. ‘Aws outpost’, provides a way to run Aws Services in the On-premises data center.
13. ‘Aws Serverless Application Repository’, provides a way to discover deploy and publish Serverless applications.
-> Now categorizing these into three different modules:
1. IAAS (Instance-based)
Examples: Amazon EC2 and virtual machines.
2. Serverless Computing (function-based)
Example: Aws Lambda
3. Container computing (Instance-based)
Examples: Amazon ECS, EKS, ECR, and Fargate.
4. Paas
Example: Aws Elastic Beanstalk.
Amazon EC2
-> It provides virtual machines in the cloud.
-> Full control over, the guest OS on each instance.
-> We can launch instances of any size into any availability zone.
-> Launch instances from Amazon machine images (AMI)
> We can control traffic to and from instances by using the Security groups.
1. ‘AMI’, is a virtual template used to create an EC2 instance. It has many Choices like (quickstart, MY AMI, Aws Market Place, and Community AMIs) it is also called a gold image in Aws. It provides disaster recovery Solutions for EC2 instances.
2. ‘Instance type’, will help us determine the memory (RAM), processing power (CPU), Disk space and disk type (Storage), and Network Performance.
-> It has various categories like General purpose, compute optimized, memory and storage optimized, etc. And offers family, generation, and Size identifiers like in (t3.large) where,
T is the family name
3 is the generation number
large is the size.
3. ‘Network features & settings’
-> The network bandwidth varies by instance type.
-> To maximize networking and bandwidth performance
1. Launch them into a cluster placement group.
2. Enable enhanced networking & network ACLs are used to Protect EC2 instances.
-> Elastic network Adapter, supports network speeds of up to 100 Gbps.
-> Intel 82599 virtual function interface, supports network speeds of up to 10Gbps.
-> EC2 instances can be launched in multiple availability zones within a region to provide fault tolerance and high availability.
-> ‘placement groups’, are used to control how EC2 instances are placed within an available zone.
4. ‘IAM Role’, is mainly used only when the software on the EC2 instance needs to interact with other Aws Services. The IAM role is attached to an EC2 instance in an instance profile
5. ‘user data’, we use the user data scripts to customize the run time environment of our instance, and the Script executes the first time the instance starts.
6. ‘Storage options’, we can configure the root volume, where AMI might already include more than one volume among which we need to specify:
-> The Size of the disk
-> The volume type
-> Encryption should be used.
-> Amazon EBS
-> Amazon EC2 instance store.
-> Mount an Amazon EFS
-> Connect to Amazon S3
7. ‘Tags’, these are labels that you can assign to an Aws resource. It is the process of tagging through which metadata is attached to an EC2 instance.
8. ‘Security Groups’, is a set of firewall rules that control traffic to the instance and it creates rules, that specify the source and which ports that network communications can use.
9. ‘Key Pair’, At each instance launch, we need to specify the existing key pair (or) create a new key pair
A key pair Consists of:
-> A public key that Aws stores
-> Private key file that you store.
Costing on EC2 Instances is Based On
1. Instance type
2. AMI type
3. Region.
-> Launching an Ec2 Instance with Awg CLI, where the EC2 instances can also be created programmatically.
-> Rebooting an instance will not change any IP address or DNS hostname
-> When an instance is stopped and then started again:
The public IPv4 address and external DNS hostname will change and correspondingly the private IPv4 address and internal DNS hostname don’t change.
->If we require a public IP address we use an Elastic IP address because,
It can be associated with instances in the Region as needed and remains allocated to our account until we choose to release it.
-> Instance metadata is data about our instances.
Amazon Cloud Watch for Monitoring
-> Provides near real-time metrics
-> provides charts in the Amazon EC2 console
-> maintains 15 months of historical data.
Amazon EC2 Pricing Models
1. on-demand instances (low cost and Flexibility)
-> pay by the hour, no long-term commitments (but now pay-per-second Pricing is applied for newer versions).
2. Dedicated hosts (compliance and regulatory requirements)
-> Physical Server with Es instance capacity
3. Dedicated Instances
-> Instances that run in a vpc on hardware only to a single customer
4. Reserved Instances (Compute capacity is available when needed)
-> Full, partial, no upfront payments with discount on hourly charge and has predictable usage patterns
5. Schedule Reserved Instances
-> Purchase a capacity reservation that is always available
6. Spot Instances (Large Scale, Dynamic workload, and availability of ec2 instances is always flexible)
-> Instances run as long as they are available, with interrupt options including terminated, stopped, or hibernated.
-> If any project requires us to run monthly reports that iterate through very large amounts of data we consider purchasing “Scheduled Reserved Instances”
Four Pillars of Cost Optimization
1. Right size
2. Increase Elasticity
3. Optimal pricing model
4. optimize storage choices
Measure, Monitor, and Improve Recommendations
1. Define and enforce cost allocation tagging
2. Define metrics, Set targets, and review regularly.
3. Encourage teams to architect for cost.
4. Assign the responsibility of optimization to an individual or team
Container Services
-> Containers are a method of operating System virtualization.
-> Containers share a virtualized operating system and run as resource-isolated processes.
-> Containers are repeatable, Self-contained environments and the Software runs the same in different environments, faster to launch and stop or terminate than virtual machines
Docker
-> Docker is a software platform that enables you to build, test, and deploy applications Quickly through which we can run containers on docker
-> Containers are created from a template called docker images.
-> Containers have everything a software application needs to run including libraries, system tools, and application code.
-> Virtual machine runs on Hypervisor, but containers run on Operating System.
Amazon Elastic Container Service( ECS)
-> A highly Scalable, fast, container management Service that orchestrates the running of Docker containers
-> Maintaining and Scaling the fleet of nodes that run on Containers and removes the complexity of standing Infrastructure.
-> Similar to EC2 it has some features like Elastic Load Balancing, Amazon EC2, Security groups,
-> Amazon EBS volumes, and IAM roles
-> Do you want to manage the Amazon EC2 cluster that runs on containers
If Yes, create an Amazon ECS cluster backed by Amazon EC2 i.e provides more granular control over infrastructure
If No, create an Amazon EC2 cluster backed by Aws Fargate i.e. easier to maintain, focus on our applications
Kubernetes
-> Kubernetes is an open Source software for container coordination and it is an alternative for Amazon ECS.
-> Deploy and manage containerized applications, with the Same toolset that can be used on premises and in the cloud.
-> Docker enables us to run multiple containers on a single OS host and Kubernetes Co-ordinates multiple docker hosts
-> Kubernetes is also capable of automating Container provisioning, networking, Load distribution, and Scaling.
Amazon Elastic Kubernetes Service (EKS)
-> manage clusters of amazon EC2 compute instances.
-> Run containers that are coordinated by Kubernetes on those Instances.
-> Supports, Linux and Windows containers and compatible with Kubernetes community tools and Supports Popular Kubernetes add-ons.
Amazon Elastic Container Registry (ECR)
-> Amazon ECR is a fully managed Docker container registry that makes it easy for developers to store, manage, and deploy Docker container images
-> It has Amazon ECS Integration, Docker Support, Team Collaboration, Access Control, and Third-party integrations.
Aws Lambda
-> It is a Serverless Compute Service, which will run the code without Servers.
-> It will run our code on a schedule or in response to events and the code will run only when it is triggered.
-> And we have to pay only for the computing time that we we It also Supports multiple programming languages and has completely automated administration with built-in fault tolerance along with pay-per-use pricing.
-> Lambda function is a custom code that we write to process events and Lambda runs the lambda function on our behalf. Aws Lambda Quotas is as follows:
1. Concurrent executions = 1000
2. Function and layer storage = 75GB.
3. Max function memory allocation=10,240MB
4. Container image code package size = 10GB.
5. Function time out (max) = 15 minutes.
6. Deployment package size 250MB
-> Aws Lambda pricing
1. The time it takes for the Aws Lambda function to execute
2. Number of requests for Aws Lambda function.
Aws Elastic Beanstalk
-> It provides an easy way to get web applications up & running
-> It is a managed service, that handles:
1. Infrastructure Provisioning and configuration
2. Deployment, Load balancing and Automatre Sealing
3. Health monitoring, analysis and debugging, Logging.
4. pay-per-use
Elastic Beanstalk automatically handles the deployment on Servers like Apache, NGINX, passenger, puma, and IIS
-> It has developer productivity, is difficult to outgrow, and Complete resource Control.
-> Aws is more economical than traditional data centers for applications with varying compute workloads because Amazon EC2 instances can be launched on-demand when needed
Amazon EC2 Auto Scaling
-> Add or remove compute capacity to meet changing demand
1. Improve fault tolerance
2. Increase availability with the right amount of computing capacity
3. Optimize workload Performance and cost
4. Reduce the complexity of configuration and application deployment.
In an EC2 Instance:
1. Horizontal Scaling: Increase capacity by adding more computers to the system
2. Vertical scaling: one computer in the entire system so we add more resources like CPU, memory, and Storage in a Single Computer system.
— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —
Congratulations! You’ve explored the diverse compute services offered by AWS and gained a solid understanding of running applications on the cloud. This knowledge equips you to select the most appropriate compute solution for your specific needs.
Leave a comment below with any questions or cloud computing concepts you’d like to explore further!
In the next module, Module 7: Storage, we’ll dive deeper into the world of AWS storage solutions. Get ready to discover various storage options for different use cases and explore strategies for optimizing your cloud storage footprint!
Episode 7: https://medium.com/@harshithavineni81/mastering-the-cloud-module-7-storage-1141ff97043b
