AWS Fundamentals: Beginners Guide

Definitive Guide to understand AWS Fundamentals.

What’s Cloud?, the son asked. It’s somebody else’s computer, the father answered. So in AWS cloud, computers belong to Amazon and we use them.

Be it cloud, or your own office. Any type of infrastructure which serves a useful purpose only consists of these three things — Compute, Storage and Network. Only if these three are properly designed and operated, something wonderful can happen. Let us see how AWS created something wonderful.

Let us go through their architecture first and dive into the internals later on.

Architecture:

AWS says it has over a million customers as of 2020. It handles everyone of them separately based on their accounts. Each customer in AWS will have their own account or accounts for that matter. So the first element in the architecture will be an ACCOUNT.

Lets say your company has customers across multiple countries and you have a primary application which will be actively accessed by your customers. One of your customer says they’re facing latency issues when accessing the application and another customer says their Data stored on AWS cloud should reside in their country only according to their policies. To deal with this, AWS launched its cloud in multiple regions across the world. As of 2020, there are 24 regions. So you can have your application running in your customer’s region or as close as possible to his region to minimize latency issues and deal with compliance policies. You can do all of this using the same account. So the second element is REGION.

In a region you launch your actual application. You will have your own logical data center given to you by default, known as VPC — Virtual Private Cloud. This local data center or VPC will be completely under your control. Think of it like a new on-premise branch of yours. You can completely build your compute, storage and network from scratch inside your VPC. More on VPC in coming sections. So the third element would be a VPC.

Till now it’s all good, let’s now talk something negative. Let’s say most of your customers are from N.Virginia and you have launched a really great application stack in that region, in your VPC. Unfortunately, the place where the real data center in N.Virginia is, got hit by a terrible earthquake and DC is no more operational. What next? No one to blame in this case. To deal with these kind of problems AWS came up with multiple zones in a region simply known as an Availability Zone. So the fourth element in the architecture is — Availability Zone.

An availability zone may have multiple Physical Data Centers with it’s own power supply, cooling and physical security. All the AZs in a region are connected via redundant and ultra low-latency networks. Availability zones allow us to achieve proper fault tolerance. Number of availability zones vary from region to region. AWS considers factors like power distribution, floodplains, and tectonics when placing Availability Zones within a region so that each availability zone has a different risk profile. Availability Zones are physically separated by a meaningful distance from each other, although all are within 100 km (60 miles).

Inside the availability zone is where we will create actual sub-networks — considering VPC is our entire big network, we can create multiple sub-networks based on the requirements to make the network as efficient and secure as possible. So Subnet is the fifth element in the architecture.

In terms of architecture we are done, as the actual Servers where any application or database or anything for that matter is hosted resides in a Subnet. If i put this whole explanation in a picture, it looks something like this —

AWS basic architecture. Made on draw.io

Check this out to understand AWS Global Infrastructure — https://www.infrastructure.aws/

Let us now discuss about the core compute, storage and network components which are the primary building blocks to make your infra functional. I will first list them by their category —

Compute:

EC2

Storage:

I would like to classify storage into two categories — Traditional File System(TFS) and Database.

TFS:

EBS

EFS

ECR

S3

Database:

In Databases, there are many services that AWS offers, we’ll only discuss some of them —

RDS(Relational Database Service)

DynamoDB

Redshift

Network:

The primary network components in your account are the following—

IGW — Internet Gateway

Implicit Router(Virtual)

Route Tables

NACL — Network Access Control Lists

NAT Gateway

Security Groups

All the network components in this list exists by default when we create an account, in the VPC except for NAT Gateway. We can spin up a NAT Gateway or a NAT instance according to the needs. More on NAT Gateway/Instance in coming sections.

If all the listed components are put in an image, it will look something like this —

AWS Network with its core components. Made on draw.io

Now let’s understand what each component does in this network.

Compute:

EC2:

EC2 stands for Elastic Compute Cloud. It is the core compute component in the infra. This is nothing but a simple server where applications can be hosted. There are multiple types of EC2s available from AWS, each type serving a slightly different purpose. AWS uses the word EC2 Instance referring to a server.

Almost all AWS services will have a prefix called ELASTIC meaning those services can scale or shrink in matter of seconds.

EC2 Types:

There are different types of EC2 instances each serving a slightly different purpose. Lets say if you host a gaming server, you expect a lot of network speed than any other thing, and if you are implementing some really complicated ML, DL algorithm, you would expect super computing power than any other thing, based on the usage you can choose the type. These are the types of EC2 instances each denoted with different combinations of alpha-numeric characters —

1. General Purpose — A1, T3, T3a, T2, M6g, M5, M5a, M5n, M4. This type is a proper balance of compute, storage and network.
2. Compute Optimized — C6g, C5, C5a, C5n, C4. As the name suggests, primarily used for High Performance Computing activities like Dedicated Gaming Servers, High Batch Processing etc.
3. Memory Optimized — R6g, R5, R5a, R5n, R4, X1e, X1, High Memory, z1d. Ideal for workloads which process large sets of data.
4. Accelerated Computing — P3, P2, Inf1, G4, G3, F1. These instances are powered by Hardware Accelerators and are suitable for Machine Learning, Deep Learning workloads.
5. Storage Optimized — I3, I3en, D2, H1. Ideal when working on huge sets of Data which will require very high sequential read and write access such as Data Warehousing, Data Analytics activities.

Each Type is available in different sizes. We can choose the type and size of an instance. The available sizes are — nano, micro, small, medium, large and xlarge(extra large). Please note that all types of instances are not available in all sizes. Some instances start from nano and some start directly from medium. For example, if a memory optimized instance is offered with nano as the size of the instance it wouldn’t make any sense.

EC2 Pricing Model:

Pricing of EC2s is simply divided into four categories which are —

1. On-Demand — The most commonly used ones. Can start and terminate whenever required and you are charged on an hourly basis.Balanced Pricing with Least Commitment.
2. Spot Instances — Cheapest in the list. If you are working on non-critical tasks and ready for some interruptions, you are good to go. AWS claims Spot instances are upto 90% lesser than On-Demand in terms of cost.
3. Reserved Instances — These are best for long-term. If you can commit to using an EC2 instance over a 1 or 3 year term, this is the best choice. AWS claims you get upto 75% of discount compared to On-Demand if you use Reserved Instances.
4. Dedicated Hosts — Most expensive in the list is this one. You will have a physical EC2 server dedicated for your use. If you need a guarantee of isolate hardware you can go with this. Some enterprise level companies may have security obligations about sharing same hardware with other customers.

EC2 Placement Groups:

Placement Groups allows us to choose the logical placement of instances to make sure all the interdependent instances are properly placed to meet our workload needs. Available Placement group types are —

1. Cluster — Instances are closely packed which helps in keeping the latency as low as possible. It is suited for High Performance Computing Applications.
2. Partitions — Instances are spread across multiple partitions so that groups of instances in one partition do not share the underlying hardware resources with other groups of instances in other partitions.
3. Spread — Instances are spread across distinct underlying hardware to ensure there are no correlated failures. When instances should be strictly kept separate from each other, this could be your option.

Placement Groups are Free of Cost.

Storage:

TFS:

EBS — EBS stands for Elastic Block Storage. You can think of a normal hard drive which can be attached to your Computer. When you launch an EC2 instance, you are supposed to choose the size of the EBS volume. You can choose a size starting from 1GiB to 16TiB. This is for non-root volumes, for root volumes AWS sets the minimum size of the volume, you cannot go below that, as root volume contains the OS.

When talking about EBS, its worth mentioning Instance Storage. You can launch an EC2 either with an EBS or other option is Instance Storage. Instance storage is temporary — ephemeral in nature, data will be lost after instance is terminated.

NOTE: EC2s with Instance Storage cannot be stopped or rebooted as they are ephemeral in nature, they can only be terminated. But EBS backed EC2s can be rebooted, stopped and started without Data Loss.

EFS — EFS stands for Elastic File System. You can think of a Network File System. While EBS volume can be attached to one instance at a time, you can attach EFS to multiple EC2s at same time, so all the instances share this File System.

ECR — ECR stands for Elastic Container Registry. It is basically used as a Docker Container Registry. We can create repositories in ECR and maintain docker images inside a repository and pull and push them accordingly.

S3 — S3 stands for Simple Storage Service. S3 is an extremely simple cloud storage service by AWS. It stores everything as an object in buckets. S3 objects can range in size from a minimum of 0 bytes to a maximum of 5 terabytes.

Database:

RDS — RDS stands for Relational Database Service. Just like the EC2 instance types, RDS is also available on different database instance types — optimized for performance, memory or I/O. RDS provides six familiar Database Engines to choose from — Amazon Aurora, MySQL, PostgreSQL, MSSQL, MariaDB and Oracle DB.

DynamoDB — Amazon DynamoDB is a NoSQL database which basically supports key-value and document data structures. AWS claims it delivers single-digit millisecond performance at any scale and can handle more than 10 trillion requests per day.

RedShift — AWS Red Shift is basically a data warehouse service in the cloud which allows us to crunch huge amounts of data. It works as a cluster(set of nodes packed to achieve the same purpose). Redshift allows us to run high performance queries on petabytes of structured data. Primary use cases are business intelligence (BI), reporting and Operational analytics on business events.

While there are many more Database services offered by AWS, i have mentioned only a few of them just to provide a basic overview of them and not to make this article Database oriented. More on AWS Database Services — https://aws.amazon.com/products/databases/ .

Now coming to my most favorite part in the article — Network.

Network:

VPC Intro: As mentioned before VPC is one primary element which is like our own Logical Data Center which will be completely under our control. When you create a VPC in an AZ inside a region, its like you have opened a completely new branch, a logical one though. It doesn’t matter whether it’s logical or physical as long as it is operational. To create anything in the account, VPC should exist first, so we’ll start with VPC.

CIDR:

When creating a VPC, we need to specify a CIDR block which will be our primary network of the VPC. This primary network can be further split into subnets. More on subnets in the coming sections.

The biggest IPv4 CIDR block that can be created is /16 and the smallest that can be created is /28.

For example —Biggest:x.x.x.x/16 [10.0.0.0/16] & Smallest: x.x.x.x/28[10.0.0.0/28]. We cannot go below /16 and beyond /28.

This CIDR block is always considered private by AWS irrespective of the range we use, as the IPs used from this range are only used for Private communication between the instances and not for Public/Internet communication. So even if we specify 150.0.0.0/16 which is a public IP Range, AWS treats this as a private IP and you cannot reach the internet using these IPs. For public communication we can ask AWS to assign us an Public or Elastic IP to the instance. We will discuss about Elastic & Public IPs shortly.

Note: In case of IPv6, we cannot create a specific network, AWS will only assign us a CIDR block with /56 as mask. As IPv6 are GUAs — Global Unicast Addresses, there’s nothing like private range and public range, every IP is public and can be reached by anyone on the internet.

Elastic IP, Public IP and Private IP:

Elastic IP — Elastic IP is a combination of static and public IP in AWS that can be added to your account and can be assigned to instances whenever required.

Use Case: Lets say you hosted a service which should be accessible from the Internet. So you assign an elastic IP, so that this service is reachable via this EIP. In case the instance where the service is hosted went down, you can still remap it to a different instance and make the service available on the same IP.

Public IP — Unlike Elastic IP, this is dynamic Public IP. You will get a public IP to an instance which is reachable via internet. But in case you stop your instance and start it later, you would get a completely different Public IP address.

Private IP — This IP is mandatory and is used for internal communication between instances within a VPC or across multiple VPCs in case of VPC Peering is set up. More on VPC Peering in a different article. This IP is given from the CIDR range we assigned when we created the VPC.

In the network element list, only IGW — Internet Gateway resides out of VPC, every other element resides in the VPC.

As mentioned earlier, somethings in the account exist by default — a default VPC, IGW, Virtual Router, Routes, some subnets and a security group. We can delete the default stuff and create all the elements from scratch.

Now let’s start with the actual elements in the Network Section —

IGW — Internet Gateway

As the name suggests it’s a device which will allow you to reach internet. Think of your home Router/Access Point which simply does the same job — allows your devices to communicate with internet. But the only difference is this IGW is horizontally scalable, highly available unlike your home access point where you will need to restart sometimes.

You can only attach one IGW to a VPC at any time. By default the maximum no of IGWs allowed per region are 5.

Implicit Router

This is a Router in your VPC which is implicitly present — meaning you need not and of course cannot create and delete this device. This is not a device which you can directly take control of. The only way to talk to this Router is by interacting with Route Tables. Route Tables are completely under your control. We’ll talk about Route Tables just in a moment.

Route Tables

These tables are no different from your Route Tables in a Home Based Router/Access Points. These will tell how your resources in one subnet can reach the Internet or any other resources in any other subnet. A route table can be associated with multiple subnets, however one subnet can only be associated with one route table at a time. A route table in AWS looks like this —

Editing Route Table in AWS.

This is super simple. It has destination and a Target. Destination can be a single IP address or an entire CIDR block. Target is the actual resource in AWS where the traffic gets routed Target can be one of these -[IGW, Instance, Egress only IGW, NAT Gateway, Network Interface, Outpost Local Gateway, Peering Connection, Transit Gateway, Virtual Private Gateway]. Status speaks for itself. Propagated field by default says NO. If you want your VPG(Virtual Private Gateway — more on this in another article) to dynamically propagate routes into your Route Table, this field says YES.

NACL — Network Access Control List

Basically a NACL in AWS is a stateless firewall — meaning you have to explicitly tell what should be allowed and what should be denied. So there should be Set of Inbound and Outbound Rules both specifying Allowed and Denied Traffic. By default NACL allows ALL inbound and outbound traffic. Like Route Tables, NACLs can also be associated to multiple subnets, however one subnet can only be associated with one NACL at a time. A default NACL in AWS looks like this —

Default NACL Inbound Rules.

Default NACL Outbound Rules.

Either in an Inbound Rule Table or Outbound one, we have six fields to fill here which are —

Rule Number — Rules are evaluated starting with the lowest numbered rule. If a rule matches, it gets executed without checking for any other higher numbered rules.

Type — Type of Traffic which can be TCP, UDP, ICMP. Type field provides the well-used protocols, when selected it auto fills the Protocol field. You may also select Custom Protocol Rule which allows you to select the Protocol field from a wide range of Protocols.

Protocol — As mentioned already, if you select Custom Protocol Rule in Type field, you can select a Protocol from the available Protocol List.

Port Range — You can specify a single port or a range of ports like this 5000–6000.

Source[Inbound Rules only] — Can be a Single IP Address or an entire CIDR block. Destination can only be mentioned in Outbound Rule.

Destination[Outbound Rules only] — Can be a Single IP Address or an entire CIDR block. Source can only be mentioned in Inbound Rule.

Allow/Deny — Specifies whether to allow or deny traffic.

NAT Gateway

NAT stands for Network Address Translation and basically enables your Resources in a Private Subnet communicate with internet without assigning each resource a Public IPv4 Address. I specifically mentioned IPv4 address because, an IPv6 address has no concept of Private/Public Address. Each IPv6 address is a public IP and a Global Unicast Address which means anyone on the internet can reach out to this address, given there are no NACLs written to deny traffic to this address.

So the best use case of a NAT Gateway would be — when you want your resources in Private Subnet access internet, but prevent the internet from initiating a connection to those Resources.

Security Group

Unlike NACL, Security Group is a Stateful Firewall — meaning if inbound traffic is allowed by you, outbound response from your resource is automatically allowed irrespective of your Outbound Rules. Security Group can be stateful by keeping track of connections. The primary difference between a Security Group and NACL is this — Security Group is Stateful while a NACL is not, it doesn’t keep track of any connections. Security Group works at instance level while a NACL works at Subnet level. So you can think of Security Group as a Firewall to an Instance — it can be an EC2 instance or a Database instance.

Conclusion

With all this said, let us conclude the explanation of AWS Fundamentals here. This whole theory is only to open a doorway for you to go and experiment with AWS Cloud as the real understanding of any thing lies in practice and not in theory. So I am strongly recommending you to go ahead and create an AWS account for yourself and start building things.

AWS Free Tier —

If you are a first time user, AWS gives you an opportunity to learn by providing you a Free-Tier account for the first 12 months. Free Tier includes so many resources for which you are not charged. For example — AWS provides 750 Free EC2 Hours/month for the first 12 months. It doesn’t mean that you can go ahead and run a ‘m5a.16xlarge’ for 750hrs. Obviously there are some limits to what you can and cannot run in AWS under free-tier. To know more on AWS Free-Tier — https://aws.amazon.com/free/

AWS Pricing —

Resources mentioned while explaining, in the article may or may not be free. So please do get a proper understanding of Resource Pricing in AWS before you spin anything up. To know more about Pricing — https://aws.amazon.com/pricing/

AWS VPC Quotas/Default Resource Limits —

Some of these resource quotas are strict(cannot be increased) and some other can be increased by requesting AWS. To know more on VPC Quotas — https://docs.aws.amazon.com/vpc/latest/userguide/amazon-vpc-limits.html

Thank you for reading. You can connect with me on Linkedin . Happy to answer your queries.

Happy Cloud Computing Folks!!