AWS Solutions Architect all-you-need Notes
Here are short notes to revise once you are done preparing for the AWS solutions architect certification
AWS Regions and AZ
AWS Region: a cluster of data centers
Choosing AWS region :
1) compliance: government policies (Indian data cannot leave India without permission so choose the Mumbai region)
2) proximity: reduced latency
3)Availability of services
4)pricing varies on the region
Availability zones: one or more discrete data centers
-To ensure that resources are distributed across the Availability Zones for a region, AWS maps Availability Zones to names for each AWS account. For example, the Availability Zone us-west-2a for one AWS account might not be the same location as us-west-2a for another AWS account.
To coordinate Availability Zones across accounts, you must use the AZ ID, which is a unique and consistent identifier for an Availability Zone. For example, usw2-az2 is an AZ ID for the us-west-2 region and it has the same location in every AWS account.
AWS point of presence (Edge locations) :
-An AWS Edge location is a site that CloudFront uses to cache copies of the content for faster delivery to users at any location.
IAM and AWS CLI
IAM Policy structure:
1) Version
2) Id
3) Statement
The statement consists of :
-Sid
- Effect : Allow/Deny
- Principle : which account/role this policy applied to.
-Action : list of actions allowed “s3:GetObject” , “s3:PutObject”
- Resource: list of resources s3::my bucket/*
-Condition
MFA (multi-factor authentication):
-Virtual MFA device : google authenticator (phone only) or authy (support for multiple tokens on a single device)
-Universal 2nd Factor (U2F) security key device: Example yubikey
-Hardware key fob
Cloudshell : CLI on console (data persists)
IAM roles: some AWS services will need to perform actions on behalf of you, so we assign them roles
IAM security tools :
IAM credentials report (account level): lists all users of the account and the status of their credentials
IAM access advisor (user level): shows the permission granted to the user and last when accessed
EC2
-Each EC2 instance that you launch into a VPC has a tenancy attribute. Refer link: https://docs.aws.amazon.com/autoscaling/ec2/userguide/auto-scaling-dedicated-instances.html .This attribute has the following values.
1)Default 2)Dedicated 3)Host
-You can only change the tenancy of an instance from dedicated to host, or from host to dedicated after you’ve launched it
EC2 user data :
-script that will be launched during only the first boot of the instance
-Scripts entered as user data are executed as the root user, hence do not need the sudo command in the script
EC2 instance types :
General purpose : Balance between compute, memory, and networking
Compute optimized : (C series): For compute intensive tasks requiring high level processors like batch processing and gaming servers and HPC
Memory optimized : (R series): if you require to process a large dataset in memory
Storage optimized: for accessing a lot of data on local storage
Security groups:
Acts as a firewall on AWS EC2 instances
by default, all outbound traffic is authorized and inbound is blocked
Security Groups can be attached to multiple EC2 instances within the same AWS Region/VPC
Only contain allow rules
22 = SSH — secure shell to login to Linux instance
21 = FTP to upload files to file share
22 = SFTP upload files using ssh
80 = HTTP
443 = HTTPS for accessing secure websites
3389 = Remote Desktop protocol — login to windows instance
Security groups can have other security groups and can be attached to multiples instances
Security groups are Locked down to region/VPC combinations
-CANNOT have deny rules like NACL, can have only allow rules
Purchasing options of EC2 instances
1) On-demand instances
2) Reserved (Either 1 or 3 years not in between):
a) reserved instances: long workloads like a database (AZ level)
b) convertible reserved instances: long workloads with flexible instances (if you want to change the ec2 instance type over time)
c) scheduled reserved instances: need it every Thursday 3–6 pm
3) spot instances: cheap but less reliable(can lose it anytime)
4) Dedicated hosts: book the entire physical server and control instance placement ( 3 years)
5) Dedicated instance: like dedicated hosts but does not give access to the underlying hardware. Help you address COMPLIANCE REQUIREMENTS and reduce costs by allowing you to use existing server-bound software licenses.
6) Capacity reservations: allow you to reserve specific capacity in a specific AZ for any duration
-You can do a one time spot request or a persistent spot request.
-Spot requests should be canceled first and then we can stop spot instances.
Spot fleets: set of spot instances + optional on-demand instances
-Spot Fleets are set to maintain target capacity by launching replacement instances after Spot Instances in the fleet are terminated.
-We define a possible launch pool (instance type, OS, AZ )
-Spot blocks: Spot Instances with a defined duration are designed not to be interrupted and will run continuously for the duration you select. You can use a duration of 1, 2, 3, 4, 5, or 6 hours.
Storage for EC2 instances
EBS
EBS (elastic block store) volume is a network drive(and not a physical drive and hence some latency ) that you can attach to your instance while they run.
They are bound to a specific availability zone. To move a volume to diff availability zones or region you first have to snapshot it.
-AZ level
-Can create an archive and recycle bin for snapshots
Important : By default, root EBS volume is deleted on termination.
by default, any other EBS volume is not deleted.
EBS volume type:
gp2/gp3 (SSD): general purpose SSD with balance between price and performance
-It supports max IOPS/Volume of 16,000.
io1/ io2 (SSD): highest Performance SSD for mission critical low latency and high throughput
st 1 (HDD): low cost HDD for frequently accessed, throughput intensive workload
-It supports max IOPS/Volume of 500.
sc 1 (HDD): lowest cost HDD for less frequently access workload
-It supports max IOPS/Volume of 250.
Only gp2/gp3 and io1/io2 can be used as boot volume
Difference between gp2/gp3: in gp3 we can independently set IOPS and throughput whereas in gp2 they are linked together
Io1/io2 are provisioned IOPS (PIOPS)
- For critical business applications with sustained IOPS / IOPS over 16000
- For over 32000 IOPS we will need EC2 nitro with io1 or io2
- Can attach it to more than 1 EC2 instance in an AZ
EBS io2 Block Express is the next generation of Amazon EBS storage server architecture. It has been built for the purpose of meeting the performance requirements of the most demanding I/O intensive applications that run on Nitro-based Amazon EC2 instances. With io2 Block Express volumes, you can provision volumes with Provisioned IOPS (PIOPS) up to 256,000
AMI
Amazon machine image
-Ami are customization of EC2 instances
-The AMI must be in the same region as that of the EC2 instance to be launched
-Region level
-When the new AMI is copied from region A into region B, it automatically creates a snapshot in region B because AMIs are based on the underlying snapshots
you can launch EC2 instances from:
-public AMI
-your own AMI
-aws marketplace AMI
EC2 instance store
-Block level storage like EBS
-good for cache, buffer, temporary content or for data that is replicated across a fleet of instances
-You can’t detach an instance store volume from one instance and attach it to a different instance
-When you stop, hibernate, or terminate an instance, every block of storage in the instance store is reset.
Elastic file system
-managed network file system which can be attached to 100s of EC2 hence makes it a shared file system
-EFS works ONLY with linux based AMI and not compatible with windows
-regional service
-works across multiple AZ, VPC and Regions
EFS Infrequent access (EFS IA)
-storage class cost-optimized for files NOT accessed everyday
-EFS will automatically move files to EFS-IA based on last time accessed
-You will pay a fee each time you read from or write data stored on the EFS — Infrequent Access storage class
EFS Performance modes:
General purpose(default): latency sensitive use cases like web server
Max I/O: higher latency, higher throughput
EFS Throughput modes:
Bursting: if we opt for higher storage then we get higher IO
Provisioned: set throughput regardless of storage size
-AWS recommends that you run your application in the Bursting Throughput mode. But, if you’re planning to migrate large amounts of data into your file system, consider switching to Provisioned Throughput mode
ELB and ASG
High availability: Making the system available in at least 2 availability zones
Elasticity: the presence of auto-scaling once your system is scalable
Load balancing :
Load balancers are servers that forward internet traffic to multiple servers downstream (EC2 instances in this case)
advantages :
-spread load across multiple downstream instances
-expose a single point of access (DNS)
-seamlessly handle failures of downstream instances
-do regular health checks on instances
-provide SSL termination (HTTPS) for your websites
-high availability across zones
ELB: elastic load balancer
managed load balancer (means you don’t need to provision servers, AWS will do it for you)
AWS takes care of updates, maintenance, and high availability of that load balancer
4 kinds of load balancers:
- Application load balancer (HTTP and HTTPS and WebSocket- layer 7) — supports TLS offloading
- Network load balancer (ultra high performance, allows for TCP — layer 4, UDP) (note: network load balancer cannot have a security group) — supports TLS offloading
- classic load balancer (older generation) (layer 4 and 7) — Exam will never ask to choose this since it is older — supports SSL offloading
- gateway load balancer (new — IP protocol): when you want to analyze IP traffic and send to 3rd party. The gateway load balancer uses Geneve protocol on port 6081.
For ELB, All EC2 instances need to be in the same AWS region but can be in multiple availability zones
-Network Load Balancers expose a fixed IP to the public web, therefore allowing your application to be predictably reached using these IPs
-ALB cannot use EC2 based health checks
-It is recommended to use ALB based health checks for both Auto Scaling group and Application Load Balancer instead of using EC2 based health checks for ASG.
-Only Network Load Balancer provides both static DNS name and static IP. While, Application Load Balancer provides a static DNS name but it does NOT provide a static IP.
Request Routing and IP Addresses for NLB:
-If you specify targets using an instance ID, traffic is routed to instances using the primary private IP address specified in the primary network interface for the instance.
If you specify targets using IP addresses, you can route traffic to an instance using any private IP address from one or more network interfaces. This enables multiple applications on an instance to use the same port. Note that each network interface can have its security group. The load balancer rewrites the destination IP address before forwarding it to the target.
-Between the ALB and the Elastic IP: If we use an ALB, things will still work, but we will have to pay for the provisioned ALB which sends traffic to only one EC2 instance. Instead, to minimize costs, we must use an Elastic IP
ASG scaling vs ASG rebalancing
ASG scaling: Amazon EC2 Auto Scaling creates a new scaling activity for terminating the unhealthy instance and then terminates it. Later, another scaling activity launches a new instance to replace the terminated instance
ASG rebalancing: As the Availability Zones got unbalanced, Amazon EC2 Auto Scaling will compensate by rebalancing the Availability Zones. When rebalancing, Amazon EC2 Auto Scaling launches new instances before terminating the old ones, so that rebalancing does not compromise the performance or availability of your application
Cross-zone load balancing
-Each load balancer distributes evenly across all registered instances
-By default, cross-zone load balancing is enabled for Application Load Balancer and disabled for Network Load Balancer.
-With cross-zone load balancing enabled, one instance in Availability Zone A receives 20% traffic and four instances in Availability Zone B receive 20% traffic each. With cross-zone load balancing disabled, one instance in Availability Zone A receives 50% traffic and four instances in Availability Zone B receive 12.5% traffic each
SSL/TLS
SSL cert (secure socket layer): allows traffic between client and load balancer to be encrypted in transit (in-flight encryption)
TLS (Transport layer security): the newer version of SSL
SNI (Server name indication)
SNI solves the problem of uploading multiple SSL certificates onto one web server to serve multiple websites
Clients can use SNI to specify the hostname they want to reach in the initial SSL handshake
Connection draining:
CLB -connection draining
For ALB & NLB it is called deregistration delay
Time to complete in-flight requests while the instance is deregistering or unhealthy
-Elastic Load Balancing stops sending requests to targets that are deregistering. By default, Elastic Load Balancing waits 300 seconds before completing the deregistration process, which can help in-flight requests to the target to complete. We can update this deregistration delay
Auto Scaling group (ASG):
-Scale out or scale in automatically
-Replace unhealthy instances
-Can define minimum size, desired capacity, maximum size
-will create default size number of instances
-Data is not automatically copied from existing instances to new instances. You can use lifecycle hooks to copy the data
Strategies:
Manual scaling: update ASG size manually (desired size)
Dynamic scaling: respond to changing demand
- Simple scaling: Example when cloud watch alarm is triggered (CPU > 70) then add 2
- Target tracking scaling : (I want ASG CPU to be at 40%)
- Scheduled scaling : (increase minimum capacity to 10 on Friday 5 pm)
- Predictive scaling: uses machine learning to predict future traffic and provision right number of instances
-Step scaling: When step adjustments are applied, they increase or decrease the current capacity of your Auto Scaling group, and the adjustments vary based on the size of the alarm breach.
ASG default termination policy:
-Find AZ with most instances
-If there are multiples instances in AZ choose one with the oldest configuration -> then new configuration -> then launch template
For each Auto Scaling Group, there’s a Cooldown Period after each scaling activity. In this period, the ASG doesn’t launch or terminate EC2 instances. This gives time to metrics to stabilize. The default value for the Cooldown Period is 300 seconds (5 minutes).
ASG lifecycle hooks:
You have the ability to perform extra steps before the instance goes to service
RDS, AURORA AND ELASTICCACHE
AWS RDS:
-Allows you to create DB in cloud that uses SQL as query language
-Continuous backup and restore options with point in time restore
- Daily full backups which are done during the maintenance window that we define
-Transaction logs backed up every 5 minutes
-Read replicas for read performance
-multi-AZ setup for disaster recovery
-provisioning and OS patching
-Amazon RDS Multi-AZ deployments provide enhanced availability and durability for RDS database (DB) instances, making them a natural fit for production database workloads. When you provision a Multi-AZ DB Instance, Amazon RDS automatically creates a primary DB Instance and synchronously replicates the data to a standby instance in a different Availability Zone (AZ).
Read replicas vs Multi-AZ (important) :
Read replicas :
-Can have up to 5 read replicas
-Replication is async and therefore eventually consistent
-Replicas can be within AZ, cross AZ or cross-region
-In AWS there is a cost when data moves from one AZ to another but for Read replicas you don’t pay that fee
-add their own DNS name
Multi-AZ:
-Sync replication
-Automatic failover: standby DB can be promoted as a master in case of failover. Multi-AZ means the URL is the same, the failover is automated, and the CNAME will automatically be updated to point to the standby database.
-keeps the same connection string (DNS name ) regardless of which DB is up.
-you cannot read from the standby database
Revise: changing read replicas to multi az . What happens in case of failure? Is replication still async or sync?
Upgrades to the database engine level require downtime. Even if your RDS DB instance uses a Multi-AZ deployment, both the primary and standby DB instances are upgraded at the same time. This causes downtime until the upgrade is complete, and the duration of the downtime varies based on the size of your DB instance.
-RDS applies OS updates by performing maintenance on the standby, then promoting the standby to primary, and finally performing maintenance on the old primary, which becomes the new standby
-You cannot create an encrypted Read Replica from an unencrypted master DB instance. You also cannot enable encryption after launch time for the master DB instance.
AWS aurora:
-proprietary DB(not open source) of AWS and claims to be faster than SQL and Postgres on RDS
-Will grow automatically
-up to 15 replicas compared to 5 of RDS and replication is faster
-Writer endpoint DNS name connects to the master
-The reader endpoint connects to all read replicas since read replicas also have auto scaling enabled in aurora and would be difficult for applications to keep track of where read replicas are.
-You can define a subset of read replicas with a custom endpoint instead of reader endpoint
-Supports both MySQL and PostGres
-For Amazon Aurora, each Read Replica is associated with a priority tier (0–15). In the event of a failover, Amazon Aurora will promote the Read Replica that has the highest priority (the lowest numbered tier). If two or more Aurora Replicas share the same priority, then Amazon RDS promotes the replica that is the largest in size. If two or more Aurora Replicas share the same priority and size, then Amazon Aurora promotes an arbitrary replica in the same promotion tier.
Aurora serverless:
-Automated database instantiation and auto scaling based on usage
-good for infrequent or intermittent or unpredictable workloads
Aurora multimaster:
-In case you want high availability for writer node in case of failover
-In this case, all nodes do reading and writing
Aurora global database:
1 primary region (read/write)
Up to 5 secondary regions (read-only) and 16 read replicas per region
-Amazon Aurora Global Database is designed for globally distributed applications, allowing a single Amazon Aurora database to span multiple AWS regions. It replicates your data with no impact on database performance, enables fast local reads with low latency in each region, and provides disaster recovery from region-wide outages.
Elasticache :
Redis vs Memcached
Redis:
-Multi-AZ with auto-failover
-Read replicas are used to scale reads and has high availability
-persistent cache
-backup and restore
-Redis sorted sets guarantee uniqueness and element ordering ( example : game leaderboard )
-Redis has purpose-built commands for working with real-time geospatial data at scale
-does not support multi-threading
Memcached:
-Multinode for partitioning data ( sharding)
-No high availability ( replication)
- not persistent
-no backup and restore
-supports multi-threading
Route 53
DNS: translates human friendly hostnames to IP addresses
Route 53:
-Highly available, fully managed, scalable, authoritative DNS.
authoritative = you can update DNS records
Route 53 records:
A record contains -
-Domain/ SubDomain name: example.com
-Record type:
-Value: 12.34.56.78
-Routing policy: How route 53 responds to queries
-TTL: the amount of time that record can be cached at DNS resolvers
Record types:
(Must know ones are) : A, AAAA, CNAME, NS
A: maps hostname to IPV4
AAAA: maps hostname to IPV6
CNAME: maps hostname to another hostname (can’t create CNAME record for top node of DNS namespace(zone apex)) Example: can’t create CNAME for example.com but can for www.example.com
NS: Name servers of the hosted zone
Hosted zones:
Container for records that will define how to route traffic to a domain and its subdomains
Record TTL:
Caching result of record with clients for given TTL
Mandatory for every record except ALIAS
ALIAS:
You should also note that Route 53 doesn’t charge for alias queries to AWS resources but Route 53 does charge for CNAME queries. Additionally, an alias record can only redirect queries to selected AWS resources such as S3 buckets, CloudFront distributions, and another record in the same Route 53 hosted zone; however, a CNAME record can redirect DNS queries to any DNS record. So, you can create a CNAME record that redirects queries from app.covid19survey.com to app.covid19survey.net.
Routing policies:
a)Simple routing policy: No health checks
-IF multiple values are returned by DNS then random is chosen by the client
b)Weighted routing policy: balanced load across different ec2 depending on the weight of each record within the weighted set.
-Health checks
c)Latency routing policy: looks where the user is located and redirects to closest LATENCY server.
-Latency is based on traffic
-German users may be redirected to the US if that’s the lowest latency
-Health checks.
d)Failover routing policy: If the primary instance fails, will be redirected to the failover instance. Health check is mandatory on first instance. ( active-passive failover configuration )
e)Geolocation:
Based on user location.
-Create a default record in case of no match
f)Geoproximity:
-Based on user and resource location
-You can also optionally choose to route more traffic or less to a given resource by specifying a value, known as a bias ( 1 to 99 ) pr ( -1 to -99 )
g)multi-value
-to route traffic to multiple resources
-different from simple routing policy multiple values because simple routing doesn’t allow health checks
-different from ELB as this is client-side load balancing
AWS S3
-serverless
files = objects, directories = buckets
buckets must have a globally unique name(across all regions)
but buckets are defined at the region level
So S3 has a global console but when you create a bucket it is linked to a region
Naming convention: no uppercase, no underscore, 3–63 chars, not an ip, must start with lowercase or number
-3,500 PUT/COPY/POST/DELETE or 5,500 GET/HEAD requests per second per prefix in a bucket and There are no limits to the number of prefixes in a bucket.
-objects have keys
-key is the full path from bucket name. Example : /file.txt in s3://my-bucket/file.text
-max object size is 5000GB but if uploading more than 5GB at a time, must use multi-part upload ( Recommended to use multipart for when over 100mb)
Bucket policy (important):
-using policy generator
-With bucket policies, you can grant users within your AWS Account or other AWS Accounts access to your Amazon S3 resources.
-With ACLs, you can only grant other AWS accounts (not specific users) access to your Amazon S3 resources.
AWS S3 versioning: updated at the bucket level
-protects from unintended delete
-any file not versioned prior to versioning will have version null
-suspending versions will not delete previous versions
-Once you version-enable a bucket, it can never return to an unversioned state. Versioning can only be suspended once it has been enabled.
If you don’t toggle ‘list versions’ and proceed to delete a file then it is only marked as type=“Delete marker” and can be restored by deleting the object marked as type=“delete marker”
S3 encryption :
4 methods to encrypt
i)SSE-S3: encrypts s3 using keys handled and managed by AWS (S3 supplies own data key)
-Uses AES 256
-must set header “x-amz-server-side-encryption”:“AES256”
ii) SS3-KMS: leverage AWS key management service
-must set header “x-amz-server-side-encryption”:“aws:kms”
iii) SSE-C: manage own encryption keys
-AWS does not store encryption key that you provide
iv) Client side encryption
S3 consistency model
-All operations are strongly consistent.
MFA-Delete: need to enable versioning on bucket to use this
Only the bucker owner ( = root account ) can enable/disable MFA-delete
S3 replication:
-asynchronous replication
Cross-region replication (CRR)
same region replication (SRR)
- Must enable versioning on both source and target bucket
- Must give proper IAM permissions to S3
-Only new objects are replicated. If old objects need to be replicated then use S3 batch replication. To replicate old objects use sync command.
-Permanently deleting a specific version is NOT replicated.
S3 pre-signed URLs:
-Users given a pre-signed URL inherit the permission of the person who generated the pre-signed URL for get/put
-Can set minutes or hours of validity
S3 storage classes:
i) S3 standard-general purpose: for frequent access
ii) S3 standard-Infrequent access (IA) : for disaster recovery and backups.
-AWS recommends Standard-IA storage if you need your full dataset to be readily accessible and want to automatically save on storage costs for files that are less frequently accessed.
iii) S3 intelligent tiering (when you don’t know if to use frequent or IA) : moves automatically between tiers depending on usage
iv) S3 one zone-IA (for files you can recreate over time and don’t risk losing it, data will not be replicated and stored in a single AZ) (minimum 3 AZ in other Storage classes). Also 20% less costly than standard IA
-The minimum storage duration is 30 days before you can transition objects from S3 Standard to S3 One Zone-IA.
v) Glacier instant access
vi) Glacier flexible access
vii) Glacier deep archive (if you know backups and archives can take a lot of time to retrieve) (cheapest of all S3 storage classes)
https://aws.amazon.com/s3/storage-classes/
-you can create lifecycle rules to move objects to different storage classes or move manually
Glacier automatically encrypts data at rest using Advanced Encryption Standard (AES) 256-bit symmetric keys and supports secure transfer of your data over Secure Sockets Layer (SSL) (both at-rest and in-transit)
Life cycle rules:
you can define transition actions: example move objects to glacier after 6 months
-Waterfall model for supported transitions: https://docs.aws.amazon.com/AmazonS3/latest/userguide/lifecycle-transition-general-considerations.html
S3 transfer acceleration :
To speed up upload/download into different region S3 bucket by transferring files to AWS Edge locations which will forward it to the region. Via public internet to edge location and then via private AWS network to another region.
-Transfer Acceleration takes advantage of Amazon CloudFront’s globally distributed edge locations. As the data arrives at an edge location, data is routed to Amazon S3 over an optimized network path.
S3 byte range fetches: parallelize GETs by requesting specific byte ranges.
S3 select and glacier select: filters data server side so that less network and less CPU is used.
S3 event notifications:
Example: S3:ObjectCreated
Use case: generate a thumbnail as soon as images are uploaded.
-Eventbridge, SNS, SQS, and AWS Lambda are the only supported destinations for publishing events from S3
-can sometime take upto 1 minute
S3 requester pays:
With requester pays bucket, the requester pays instead of the owner for the cost of the request and data download from the bucket.
Athena:
Serverless query service to perform analytics against s3 objects
used SQL to query
use cases: analyze VPC flow logs, ELB logs
-can have reporting attached to Athena like amazon quick sight
S3 object lock (enable versioning)
-Object retention:
i)Retention period
ii)Legal hold: no expiration date:
-Modes:
i)Governance mode: users can’t overwrite or delete objects and alter lock settings unless have special permission.
ii)Compliance mode: Nobody can alter
Cloudfront and AWS global accelerator
Cloudfront: Is a CDN
-improves read performance by caching content at edge locations
-216 points globally
-Provides protection against DDOS
-Integration with shield
-CloudFront supports HTTP/RTMP protocol based requests
-Proxy methods PUT/POST/PATCH/OPTIONS/DELETE go directly to the origin from the POPs and do not proxy through the regional edge caches
-data transfer out for content by using CloudFront is often more cost-effective than serving files directly from S3, and there is no data transfer fee from S3 to CloudFront. You only pay for what is delivered to the internet from CloudFront, plus request fees.
-You cannot attach IAM roles to the CloudFront distribution. Here you need to use an OAI.
Cloudfront origins:
i)S3 bucket
-You can enhance security between CloudFront and s3 using Origin access identity ( OAI) : this allows communication with s3 only via CloudFront
-OAI is an IAM role for the edge location using which it’ll access S3 and the s3 bucket policy will check if this role is acceptable.
ii)Custom origin (HTTP)
-can be application load balancer
-EC2 instance
-S3 website
-any HTTP backend
Note: Security group of ALB and EC2 must allow public IP of edge locations.
Cloudfront geo-restriction : can have whitelist and blacklist for countries
Cloudfront signed URL/ signed Cookie:
Signed URL: access to individual files
Signed URLs: access to multiples files
Signed URL vs S3 pre-signed URL:
Signed URL should be used when we want to distribute data with CloudFront in front of S3 or HTTP (custom origin) and S3 pre signed URL should be used for giving direct access to S3
AWS global accelerator: (revise) (important)
-AWS Global Accelerator is a network layer service that directs traffic to optimal endpoints over the AWS global network, this improves the availability and performance of your internet applications
-If application is deployed and users are global it can add a lot of latency
-As your application architecture grows, so does the complexity, with longer user-facing IP lists and more nuanced traffic routing logic. AWS Global Accelerator solves this by providing you with two static IPs that are anycast from our globally distributed edge locations, giving you a single entry point to your application, regardless of how many AWS Regions it’s deployed in. This allows you to add or remove origins, Availability Zones or Regions without reducing your application availability
-Global Accelerator is a good fit for non-HTTP use cases, such as gaming (UDP), IoT (MQTT), or Voice over IP as well as for HTTP use cases that specifically require static IP addresses or deterministic, fast regional failover.
-CloudFront is designed to handle HTTP protocol meanwhile Global Accelerator is best used for both HTTP and non-HTTP protocols such as TCP and UDP
-With AWS Global Accelerator, you can shift traffic gradually or all at once between the blue and the green environment and vice-versa without being subject to DNS caching on client devices
-If you have workloads that cater to a global client base, AWS recommends that you use AWS Global Accelerator. If you have workloads hosted in a single AWS Region and used by clients in and around the same Region, you can use an Application Load Balancer or Network Load Balancer to manage your resources.
AWS Serverless
-Developers dont manage servers.
Lambdas :
-Limited by time — for shorter executions
-Run on-demand
-scaling is automated
- pay per request and per compute time(duration)
- Event-driven: functions only get invoked by AWS when needed. This makes it a reactive service.
- 15 mins time limit per execution
-By default, Lambda functions always operate from an AWS-owned VPC and hence have access to any public internet address or public AWS APIs. Once a Lambda function is VPC-enabled, it will need a route through a NAT gateway in a public subnet to access public resources:You should only enable your functions for VPC access when you need to interact with a private resource located in a private subnet. An RDS instance is a good example.
Once your function is VPC-enabled, all network traffic from your function is subject to the routing rules of your VPC/Subnet. If your function needs to interact with a public resource, you will need a route through a NAT gateway in a public subnet.
-AWS Lambda currently supports 1000 concurrent executions per AWS account per region. If your Amazon SNS message deliveries to AWS Lambda contribute to crossing these concurrency quotas, your Amazon SNS message deliveries will be throttled. You need to contact AWS support to raise the account limit. Therefore this option is correct.
Lambda limits:
per region
Max execution time is 15 minutes
4kb environment variable
Disk capacity /tmp 512 mb
Lambda deployment size compressed 50mb, uncompressed 250mb
Can use /tmp folder to load big files at startup
Lambda@Edge
Deploy lambda function alongside your CloudFront CDN
Dynamo DB:
fully-managed highly available distributed database with replication across multiple AZ
NoSQL database: key-value database
serverless database
use for single digit millisecond low latency
Standard and Infrequent Access (IA) table class
Each table can have an infinite number of items (rows)
Max item size is 400kb
Datatypes: scalar types, document types, set types
Reads can be eventually consistent or strongly consistent
Con: can only query on the primary key, sort key, indexes
Read and write capacity modes (important)
Two modes:
1)provisioned mode: You specify read/writes per second. Read capacity units and write capacity units (RCU and WCU). Possibility to add auto scaling for RCU and WCU
2)on-demand
DynamoDB accelerator — DAX
fully managed in-memory cache for DynamoDB
No way to join with another table
5 mins TTL
DynamoDB streams:
time-ordered stream of item-level modification(create/update/delete)
-By default, all DynamoDB tables are encrypted under an AWS-owned customer master key (CMK), which does not write to CloudTrail logs
DynamoDB global tables:
way to make DynamoDB table accessible with low latency in multiple regions
active-active replication: means you can read and write to any AWS region
-DynamoDB global tables replicate data automatically across your choice of AWS Regions and automatically scale capacity to accommodate your workloads
-DynamoDB Streams needs to be enabled first before you create a DynamoDB Global Table
API gateway:
-Lambdas are not exposed as an API directly, we need to expose them via API gateway
-Amazon API Gateway is a fully managed service that makes it easy for developers to create, publish, maintain, monitor, and secure APIs at any scale.
-serverless
-supports REST and web sockets
-can handle different environments (dev, test, prod)
-handle security (authentication and authorization)
-can cache api responses. The default TTL value for API caching is 300 seconds. The maximum TTL value is 3600.
-can add rate limiting
-Can also expose any AWS api not just lambda
-To prevent your API from being overwhelmed by too many requests, Amazon API Gateway throttles requests to your API using the token bucket algorithm
-Throttling is the process of limiting the number of requests an authorized program can submit to a given operation in a given amount of time
-If the IAM role that you create for the Lambda function is in the same AWS account as the bucket, then you don’t need to grant Amazon S3 permissions on both the IAM role and the bucket policy. Instead, you can grant the permissions on the IAM role and then verify that the bucket policy doesn’t explicitly deny access to the Lambda function role. If the IAM role and the bucket are in different accounts, then you need to grant Amazon S3 permissions on both the IAM role and the bucket policy.
API Gateway security (important)
I) IAM permissions: to give one of the users of role access to API
-leverages sig v4 (IAM credentials are in one header and passed on to API gateway)
ii)Lambda authorizer (formerly custom authorizer)
-Uses AWS Lambda to validate token In your header being passed
-can cache the result of authentication
-helps to use OAuth/ SAML/ 3rd party authentication
-Lambda must return the IAM policy for the user
ii)Cognito user pools:
-API gateway verifies identity automatically from AWS Cognito
-Cognito only helps with authentication but NOT authorization
AWS Cognito:
When we want to give a user an identity
I)Cognito user pools (CUP):
-serverless database of users for mobile apps
-username password combination
-possible to verify email, phone, MFA
-provide built-in user management
-can enable federated identities (fb, google, saml)
-sends back JWT
ii)Cognito identity pools(Federated identity):
-Provide AWS credentials to users so that they can access resources directly
-Login to federated identity pool (FIP) using a token from CUP to verify. Once verified, FIP will talk to the STS service to get temporary credentials which is passed on to the app.
iii)Cognito sync:
-Store preferences, config, state of App
-Deprecate, use aws app sync now
-cross-device sync
-offline capabilities
-requires FIP
Serverless application model (SAM):
-a framework for developing serverless applications
-all config in YAML
-SAM can use codedeploy to deploy lambda
Disaster recovery
Recovery point objective : data loss between disaster and point to which you can recover
Recovery time objective : downtime from disaster to recovery
Strategies in order of faster RTO ( less downtime)
1)Backup and restore (High RPO -High RTO)
2)Pilot light: a small version app is always running on cloud and useful for critical core(pilot light)
3)Warm standby
4)Hot site/ multi site: Full production scale is running on both AWS and on-prem
Database migration service (DMS) : on premises to AWS migration
-Also supports heterogeneous migration : SQL to Aurora
-Continuous data replication using CDC (change data capture )
AWS schema conversion tool: If source and target DB don’t have same engine
Aws application discovery service: to gather info about on-prem servers to plan a migration and track with AWS migration Hub
AWS server migration sevice (SMS): incremental replication of on premises live servers to AWS
AWS datasync : to move large amount of data on-prem to AWS
-AWS DataSync is an online data transfer service that simplifies, automates, and accelerates copying large amounts of data to and from AWS storage services over the internet or AWS Direct Connect
-as name suggests, syncs data to S3,EFS,FSx for windows
-AWS DataSync supports only NFS and SMB file types
AWS VPC
CIDR range: range of IP addresses allowed within your VPC
Two components in CIDR
Base IP: Represents IP contained in the range. Ex: 192.168.0.0
Subnet mask: Defines how many bits can change
Private IP can allow only certain values :
10.0.0.0/8
172.16.0.0/12 — AWS default VPC is in this range
192.168.0.0/16
-You can haven max 5 VPC in an AWS region
-Max CIDR per VPC is 5
for each CIDR :
Min size is /28
Max size is /16
subnet: partition of network-associated with availability zone
-tied to an AZ
-A subnet is a range of IP addresses within your VPC. A subnet spans only one Availability Zone in the Region.
-Subnet CIDR cannot go beyond VPC CIDR
-AWS reserves 5 IP in each subnet (first 4 and last 1)
(Important )If you need 29 IP addresses you can’t choose subnet of size /27 since these are 32 IP addresses and 5 are reserved. So have to use /26
public subnet: accessible to internet
-we can place ec2, load balancer etc here
private subnet: not accessible publicly
-we can place database here
-A VPC spans all of the Availability Zones in the Region whereas a subnet spans only one Availability Zone in the Region
-Internet gateways allow VPC instances to connect with internet but route tables must also be edited
Bastion host: EC2 instance in public subnet which allows us to connect to instance in private subnet through SSHing into bastion
NAT instances (outdated) : You can use a network address translation (NAT) instance in a public subnet in your VPC to enable instances in the private subnet to initiate outbound IPv4 traffic to the Internet or other AWS services, but prevent the instances from receiving inbound traffic initiated by someone on the Internet.
NAT gateway:
-AWS managed NAT instances
-will allow instances in private subnets to have access to internet while being private
To have private DNS with route 53 we need to enable two things :
DNS Resolution (enableDnsSupport)
-queries AWS DNS server at 169.254.169.253 or reserved IP at base of VPC network range
DNS hostnames : assigns public hostname to ec2 instance if it has public IPV4
NACL : network ACL
-firewall which controls traffic from and to subnet
-attached at subnet level
-can have both allow and deny rules as compared to security groups(at EC2 instance level) having only allow rules
NACL is stateless: return traffic must be explicitly allowed compared to stateful security group
-Security Group acts as a firewall at the instance level whereas Network Access Control List acts as a firewall at the subnet level(one or more subnets)
Default NACL (important)
-New subnets are assigned default NACL
-Allows all inbound/outbound
Ephemeral port:
Client connects to defined port and expects response on ephemeral port
VPC reachability analyzer : checks whether config is proper
VPC peering: to connect to another VPC via private network
-not transitive
-you must also update route tables so that EC2 instances can communicate with each other
VPC endpoints: for accessing your AWS services privately from VPC-A VPC endpoint enables you to privately connect your VPC to supported AWS services and VPC endpoint services powered by AWS PrivateLink without requiring an internet gateway, NAT device, VPN connection, or AWS Direct Connect connection
An Interface Endpoint is an Elastic Network Interface with a private IP address from the IP address range of your subnet that serves as an entry point for traffic destined to a supported service.
A Gateway Endpoint is a gateway that you specify as a target for a route in your route table for traffic destined to a supported AWS service. The following AWS services are supported: Amazon S3 and DynamoDB.
VPC flow logs:
information about IP traffic going into interfaces
-can be sent to s3 or cloudwatch
site to site VPN : to connect on premises to AWS VPC
-also known as IPsec VPN connection
-Customer gateway(CGW) : on premises must have CGW and AWS will need virtual private gateway (VGW) to use site to site VPN
-If customer gateway is public we can directly communicate between VGW and CGW or else there should be a NAT device with public IP on on-prem for communication (important)
-if you need to ping EC2 instances from on-prem , ensure ICMP protocol on inbound is present on security group (important)
AWS VPN CloudHub: can connect VGW with multiple customer networks using their own CGW and hence even customer networks can talk to each other
Direct connect(Dx) : physical connection between on premises and VPC . This is faster. This is private connection.
- AWS Direct Connect by itself cannot provide an encrypted connection between a data center and AWS Cloud, therefore we would need Direct connect plus VPN
Direct connect gateway : To setup direct connect to two or more VPC in different region
-Direct connect takes more than 1 month
-Direct connect Resiliency : (important)
1)High resiliency: one connection at multiple location
2)Maximum resiliency: TWO connections in each direct connect locations
Transit gateway : for transitive peering between thousands of VPC and on premises
-Supports IP multicast (important)
-can also be used to increase bandwidth of site to site VPN using ECMP , we can add more and more site to site VPN connection to transit gateway hence increasing bandwidth
Egress only internet gateway : NAT gateway for IPV6
-allows instances in your VPC , outbound connections over IPV6. Must update route table and prevents internet to initiate IPV6 connections
The Amazon VPC console wizard provides the following four configurations:
1) VPC with a single public subnet
2) VPC with public and private subnets
3) VPC with public and private subnets and AWS Site-to-Site VPN access
4) VPC with a private subnet only and AWS Site-to-Site VPN access
