Note : AWS Developer Certified Associate
Managing users and their level of access and their level of access to the AWS Console.
- Centralized control of your AWS account
- Shared Access to your AWS account
- Granular Permissions
- Identity Federation (including Active Directory, Facebook, LinkedIn, etc.)
- Multifactor Authentication
- Provides temporary access for users/devices and services, as necessary
- Allows you to set up your own password rotation policy
- Integrates with many different AWS services
- Supports PCI-DSS Compliance (Payment Gateway)
- Users : End users.
- Groups : A collection of users under one set of permissions.
- Roles : You create roles and can then assign them to AWS resources.
- Policies : A document that defines one (or more) permissions.
- Roles allow you to not use access key id’s and secret access keys.
- Roles are preferred from a security perspective.
- Roles are controlled by policies.
- You can change a policy on a role and it will take immediately affect.
- You can attach and detach roles to running EC2 instances without having to stop or terminate these instances.
- You can encrypt the root device volume (the volume the OS is installed on) using OS level encryption
- You can encrypt the root device volume by first taking a snapshot of that volume, and then creating a copy of that snap with encryption. You can make an AMI of this snap and deploy the encrypted root device volume.
- You can encrypt additional attached volumes using the console, CLI or API.
Amazon Elastic Compute Cloud is a web service that provides resizable compute capacity in the cloud. Amazon EC2 reduces the time required to obtain and boot new server instances to minutes, allowing you to quickly scale capacity, both up and down, as your computing requirements change.
allows you to pay a fixed rate by the time you use a instance with no commitment.
- Perfect for users that want the low cost and flexibility of Amazon EC2 without any up-front payment or long-term commitment.
- Application with short term, spiky, or unpredictable workloads that cannot be interrupted.
- Applications being developed or tested on Amazon EC2 for the first time.
provides you with a capacity reservation, and offer a significant discount on the hourly charge for an instance. 1 Year or 3 years.
- Applications with steady state or predictable usage.
- Applications that require reserved capacity.
- Users can make up-front payments to reduce their total computing cost.
It’s 2 two types for reserved
- Standard RIs (Up to 75% off on-demand) : cannot adjust any attribute of instances later.
- Convertible RIs (Up to 54% off on-demand) : feature the capability to change the attributes of the RI as long as the exchange results in the creation of Reserved Instances of equal or greater value.
- Scheduled RIs : are available to launch within the time window you reserve. This option allows you to match your capacity reservation to a predictable recurring schedule that only requires a fraction of a day, a week, or a month.
enables you to bid whatever price you want for instance capacity, providing for even greater savings if your applications have ]flexible start and end times.
- Applications have flexible start and end times
- Applications are only feasible at very low compute prices
- Users with an urgent need for large amounts of additional computing capacity.
but if Spot Instance was terminated by Amazon EC2, you won’t not be charged for a partial hour of usage.
However, if you terminate the instance yourself, you will be charged for the complete hour in which the instance ran.
Physical EC2 server dedicated for your use. Dedicated Hosts can help you reduce costs by allowing you to use your existing server-bound software licenses.
- Useful for regulatory requirements that may not support multi-tenant virtualization.
- Great for licensing which does not support multi-tenancy or cloud deployments.
- Can be purchased On-Demand (hourly)
EC2 Instance Types
Tip for remember => FIGHT DR MC PX
Amazon Elastic Block Storage (EBS)
Allows you to create storage volumes and attach them to EC2 like a harddrive on Personal Computer
EBS volumes are placed in a specific Availability Zone where they are automatically replicated to protect you from the failure to single component.
EBS Volume Types
[SSD] General Purpose SSD (GP2)
Balances price and performance for a wide variety of workloads.
- General purpose, balances both price and performance.
- Ratio of 3 IOPS per GB with up to 10,000 IOPS and the ability to burst up to 3000 IOS for extended periods of time for volumes at 3334 and above.
[SSD] Provisioned IOPS SSD (IO1)
Highest-performance SSD volume for mission-critical low-latency or high-throughput workloads
- Designed for I/O intensive applications such as large relational or NoSQL databases.
Use if you need more than 10,000 IOPS.
- Can provision up to 20,000 IOPS per volume.
[Magnetic] Throughput Optimized HDD (ST1)
Low cost HDD volume designed for frequently accessed, throughput-intensive workloads
- Big data
- Data warehouses
- Log processing
- Cannot be a boot volume
[Magnetic] Cold HDD (SC1)
Lowest cost HDD volume designed for less frequently accessed workloads
- Lowest Cost Storage for infrequently accessed workloads
- File Server
- Cannot be a boot volume.
[Magnetic] Magnetic HDD (Standard)
Previous generation for hard-drive.
- Lowest cost per gigabyte of all EBS volume types that is bootable. This are ideal for workload where data is accessed infrequently, and applications where the lowest starge cost is important
Universal command line interface for Amazon Web Services
Least Privilege : Always give your users the minimum amount of access required.
Create Groups : Assign your users to groups. Your users will automatically inherit the permissions of the group. The groups permissions are assigned using policy documents
Secret Access Key : You will see this only once. If you do not save it, you can delete the Key Pair (Access Key ID and Secret Access Key). and regenerate it. You will need to run
aws configure again.
Don’t use just one access key : Do not create just one access key and share that with all your developers. If someone leaves the company on bad terms, then you will need to delete the key and create a new one and every developer would then need to update their keys. Instead create one key pair per developer.
You can use the CLI on your PC : You can install the CLI on your Mac, Linux or Windows. I personally use S3 to store all my files up in the cloud
AWS Elastic Load Balancer (ELB)
Application Load Balancer
Application Load Balancer are best suited for load balancing of HTTP and HTTPS traffic. They operate at Layer 7 and are application-aware. They are intelligent, and you can create advanced request routing, sending specified requests to specific web servers,
Network Load balancer
Network Load balancers are best suited for load balancing of TCP traffic where extreme performance is required. Operating at the connection level (Layer 4), Network Load Balancer are capable of handling millions of requests per second, while maintaining ultra-low latencies.
Classic Load Balancer
Classic Load Balancers are the legacy ELB, You can load balance HTTP/HTTPS applications and use Layer 7 specific features, such as X-Forwarded and sticky sessions. You can also use strict Layer 4 load balancing for applications that rely purely on the TCP protocol.
if your application stops responding, the ELB (Classic Load Balancer) responds with a 504 error. This means that the application is having issues (not responding within the idle timeout period). This could be either at the web server layer or at the database layer. Identify where the application is failing, and scale it up or out where possible.
X-Forwarded For Header
You can get public IPv4 address when request passed request from load balancer by
Elasticache is a web service that makes it easy to deploy, operate, and scale an in-memory cache in the cloud. This service improves the performance of web application by allowing you to retrieve information from fast, managed, in-memory caches, instead of relying entirely on slower disk-based databases.
- Typically, you will be given a scenario where a particular database is under a lot of stress/load. You may be asked which service you should use to alleviate this.
- Elasticache is good choice if your database is particularly read-heavy and not prone to frequent changing.
- Redshift is a good answer if the reason your database is feeling stress is because management keep running OLAP transactions on it etc.
- In-memory cache sits between your application and database.
- 2 different caching strategies: Lazy load (Typically as we did) and Write Through
Lazy Loading Strategy (TTL)
- It only caches the data when it’s requested.
- Elasticache Node failures not fatal, just lots of cache misses.
- Cache miss penalty : Initial request, query database, writing to cache.
- Avoid stale data by implementing a Time To Live (TTL)
Write Through Strategy ( onChange update caching)
- Writing data into the cache whenever there is a change to the database
- Data is never stale.
- Write penalty: Each write involves a write to the cache.
- Elasticache node failure means that data is missing until added or updated in the database.
- Wasted resources if most of the data is never used.
- Object caching is your primary goal
- You want to keep things as simple as possible
- You want to scale your cache horizontally (scale out)
- You have advanced data types, such as lists, hashes, and sets.
- You are doing data sorting and ranking (such as leaderboards)
- Data persistence
- Multi AZ
- Pub/Sub are needed
What is OLTP?
OnLine Transaction Processing shortly known as OLTP supports transaction-oriented applications in a 3-tier architecture. OLTP administers day to day transaction of an organization.
The primary objective is data processing and not data analysis
What is OLAP?
OnLine Analytical Processing, a category of software tools which provide analysis of data for business decisions. OLAP systems allow users to analyze database information from multiple database systems at one time.
The primary objective is data analysis and not data processing.
RDS :: OLTP
DynamoDB :: No SQL
RedShift :: OLAP
Elasticache :: In Memory caching
Read Replica Databases
- Used for scaling, not for data redundancy.
- Mush have automatic backups turned on in order to deploy a read replica.
- You can have up to 5 read replica copies of any database.
- You can have read replicas of read replica but latency might be increase.
- Each read replica will have its own DNS endpoint.
- You can have read replicas that have Multi-AZ.
- You can create read replicas of Multi-AZ source databases.
- Read replicas can be promoted to be their own databases. This breaks the replication.
- You can have a read replica in a second region (for MySQL and MariaDB)
- Remember that S3 is Object-based: i.e. allows you to upload files.
- Files can be from 0 bytes to 5 TB.
- There is unlimited storage.
- Files are stored in Buckets.
- S3 is a universal namespace. That is, names must be unique globally.
- Read after write consistency for PUTS of new objects
- Eventual consistency for overwrite PUTS and DELETES
S3 Storage Classes/Tiers
- S3 (durable, immediately available, frequently accessed)
- S3 IA (durable, immediately available, infrequently accessed)
- S3 One Zone IA : Same as IA. However, data is stored in a single availability zone only
- S3 Reduced redundancy Storage (data is easily reproducible, such as thumbnails, etc.)
- Glacier : Archived data, where you can wait 3 — 5 hours before accessing
S3 Core Fundamentals of an S3 object
- Key (name)
- Value (data)
- Version ID
- Sub-resources (used to manage bucket-specific configuration) such as Bucker policies, ACLs, CORS, Transfer Acceleration.
- Object based storage only (for files.)
- Not suitable to install an operating system on.
- Success uploads will generate a HTTP 200 status code.
- By default, all newly created buckets are PRIVATE.
- You can set up access control to your buckets using Bucket Policies for Bucket level and Access Control Lists for object level.
- S3 buckets can be configured to create access logs, which log all requests made.
Encryption at REST
- Server side encryption : SSE-S3, SSE-KMS, SSE-C
- Client Side encryption
We can use a Bucket Policy to prevent unencrypted files from being uploaded by using creating a policy which only allows requests which include the
x-amz-server-side-encryption parameter in the request header.
- Used to enable cross origin access for your AWS resources
- By default resources in one bucket cannot access resources located in another.
- To allow this we need to configure CORS on the bucket being accessed and enable access for the origin (bucket) attempting to access.bucket URL
- Always use the S3 website URL, not the regular bucket URL.
- https://s3-eu-west-1.amazonaws.com/acloudguru — CORRECT
S3 Performance Optimization
Remember the 2 main approaches to Performance Optimization for S3
- GET-Intensive workloads : Use CloudFront
- Mixed workloads : Avoid sequential key names for your S3 objects. Instead, add a random prefix like a hash at begin of the key name to prevent multiple objects from being stored on the same partition
Read the FAQ. => https://aws.amazon.com/s3/faqs
- Edge Location : This is the location where content will be cached. This is separate to an AWS Region/AZ. Edge locations are not just READ only. you can WRITE to them too.
- Origin : This is the origin of all the files that the CDN will distribute. Origins can be an S3 buckets, an EC2 instance, an Elastic Load Balancer,(ELB) or Route53.
- Distribution : This is the name given the CDN, which consists of a collection of Edge Locations. There are 2 types. Web Distribution typically used for websites. RTMP used for media streaming
- Objects are cached for the life of the TTL (Time To Live.).
- You can clear cached objects, but you will be charged. (Invalidation)
- Lambda scales out (not up) automatically.
- Lambda functions are independent, 1 event = 1 function.
- Lambda is serverless
- Know what services are serverless!!
- Lambda functions can trigger other lambda functions, 1 event can = x functions if functions trigger other functions
- Architectures can get extremely complicated, AWS X-ray allows you to debug what is happening.
- Lambda can do things globally, you can use it to ba up S3 buckets to other S3 buckets etc.
- Remember what API Gateway is at a high level.
- API Gateway has caching capabilities to increase performance.
- API Gateway is low cost and scales automatically.
- You can throttle API Gateway to prevent attacks.
- You can log results to CloudWatch.
- If you are using JS/AJAX that uses multiple domains with API Gateway, ensure that you have enabled CORS on API Gateway.
- CORS is enforced by the client.
Version Control with Lambda
- Can have multiple versions of lambda functions.
- Latest version will use $latest.
- Qualified version will use $latest, unqualified will not have it.
- Versions are immutable (Cannot be changed).
- Can split traffic using aliases to different versions.
- Cannot split traffic with $latest, instead create an alias to latest.
AWS Step Function
- Great way to visualize your serverless application. (Lambda)
- Step Functions automatically triggers and tracks each step.
- Step Functions logs the state of each step so if something goes wrong you can track what went wrong and where.
- Tracing service in AWS
- Interceptors to add to your code to trace incoming HTTP requests
- Client handlers to instrument AWS SDK clients that your application uses to call other AWS services
- An HTTP client to use to instrument calls to other internal and external HTTP web services
- Supporting Java, Go, Node.js, Python, Ruby and .NET
- Amazon DynamoDB is a low-latency NoSQL database.
- Consists of Tables, Items (Row) and Attributes (Column).
- Supports both document and key-value data models.
- Supported document formats are JSON, HTML, XML.
- 2 types of Primary Keys : Partition Key and Composite Key (Partition Key + Sort Key).
- 2 consistency models are Strong consistent and Eventually consistent
- Access is controlled using IAM policies.
- Fine grained access control using IAM condition parameter.
dynamodb:LeadingKeysto allow users to access only the items where the partition key value matches their user ID.
- Indexes enable fast queries on specific data columns.
- Give you a different view of your data based on alternative Partition / Sort Keys
DynamoDB : Scan vs Query
- A Query operation finds items in a table using only the Primary Key attribute.
- You provide the Primary Key name and a distinct value to search for Query.
- Query result are always sorted by the Sort Key (if there is one.) that sorted in ascending order.
- Set ScanIndexForward parameter to false to reverse sorting.
- A Scan operation examines every item in the table. By default, returns all data attributes.
- Use the ProjectionExpression parameter to refine the results.
- That means you’ll fetch all data at the first time you query.
- Reduce the impact of a query or scan by setting a smaller page size which uses fewer read operations.
- Isolate scan operations to specific tables and segregate them from your mission-critical traffic.
- Try Parallel Scan rather than the default Sequential Scan.
- Avoid using Scan operations if you can design tables in a way that you can use the Query, Get, or BatchGetItem APIs.
DynamoDB Provisioned Throughput
- Provisioned Throughput is measured in Capacity Units.
- 1 x Write Capacity Unit = 1 x 1 KB Write per second.
- 1 x Read Capacity Unit = 1 x 4 KB for Strongly Consistent Read per second.
- 2 x Read Capacity Unit = 2 x 4 KB Eventually Consistent Reads per second.
Calculate Provision Throughput
Calculate Write Capacity Requirements : 100 items x 512 byte per second
- Calculate how many Capacity Units
Size of each item ÷ 1KB for Write Capacity Units => 512 bytes/ 1 Kb = 0.5 Write Capacity Units
2. Rounded-up to the nearest whole number : 0.5 => 1 Write Capacity Units per items
3. Multiplied by total items per second = 1 x 100 = 100 Write Capacity Units required
Calculate Read Capacity Requirements : 80 items x 3 KB per second
- Calculate how many Capacity Units
Size of each item ÷ 4KB for Read Capacity Units => 3kb / 4 Kb = 0.75 Read Capacity Units
2. Rounded-up to the nearest whole number : 0.75 => 1 Read Capacity Units operation
3. Multiplied by total items of read per second = 1 X 80 = 80 Read Capacity units required for Strongly Consistent, but if Eventual consistency is acceptable, divide by 2 = 40 Read Capacity Units required
- Provides in-memory caching for DynamoDb tables
- Improves response times for Eventually Consistent reads only.
- You point your API calls to the DAX cluster instead of your table.
- If the item you are querying is on the cache, DAX will return it; otherwise, it’ll perform an Eventually Consistent GetItem operation to your DynamoDB table.
- Not suitable for write-intensive applications or applications that require Strongly Consistent reads.
AWS Key Management Server (KMS)
makes it easy for creating and control the encryption keys used to encrypt your data. AWS KMS is integrated with other AWS services like EC2, S3, RDS etc. to make it simple to encrypt your data with encryption keys that you manage