Mastering the Cloud: Module 7-Storage

Amazon Elastic Block Store (EBS)

-> It Provides persistent individual block Storage volumes for use with Amazon EC2 instances.

-> It is a persistent data storage device that retains data after Power shut down (non-volatile) & can be mounted one instance at a time

-> Each Amazon EBS volume is automatically replicated within Its availability zone having high availability & Durability. with consistent and low latency performance to run workloads.

-> Amazon EBS allows us to Scale up or down resources with pay-as-you-go pricing and offer block-level storage.

Block v/s Object Storage

1. EBS Volumes Store data in a single Availability zone

2. EFS file systems store data across multiple Availability zones

-> With Block Storage, you change only the block that contains the character (Faster, uses Less bandwidth, more costlier).

-> With object storage, we have to update the entire file.

-> It can be backed up automatically to Amazon S3 through Snapshots and replication of EBS volumes is done in the same Availability zone

Uses

1. Boot volume and storage for Amazon EC2 instances.

2. Database hosts and Enterprise applications.

3. Data storage with a file system.

4. Encryption and Elasticity.

Volumes, IOPS, and Pricing

1. Amazon EBS volumes persist independently from Instance. All Volume types are charged by the amount that is provisioned Per month.

2. IOPS:

-> General purpose SSD:

Charged by the amount that you provision in GB per month until Storage is released.

-> Magnetic:

charged by the volume having no or requests.

-> provisioned IOPS SSD:

charged by the amount that you provision in IOPS

-> Snapshots:

The added cost of Amazon EBS snapshot to Amazon S3 is per GB- month of data stored.

-> Data transfer:

Inbound is free and outbound data transfer across regions Includes charges

Amazon S3

1. It is a managed cloud Storage Solution that is designed to Scale and the data is Stored as objects in the buckets

2. Designed for 11 9’s of durability and has granular access to buckets and objects (used for most frequently accessed data).

3. The objects can be images, videos, or Server logs we get fine-grained control over the data access by using AWS IAM and Amazon S3 Policies ( where S3 Provides durable static content with lower CPU utilization)

4. It offers a range of object-level storage classes that are designed for:

-> Amazon S3 Standard and Amazon S3 Intelligent-tiering

-> Amazon S3 standard — Infrequent Access

-> Amazon S3 one–zone Infrequent Access

-> Amazon S3 Glacier and Amazon S3 Glacier Deep Archive.

5. To upload our data into Amazon S3 in the form of objects into the S3 buckets, first we have to create a bucket in the Aws region and now we can control who can create, delete & list objects in the bucket with the help of access logs ( s3 allows you to store unlimited amounts of data but not used to store objects of virtually unlimited size)

Amazon S3 Buckets URL’s Types

1. Bucket path style URL endpoint

Example: http://s3.ap-northeast-1.amazonaws.com / bucket-name

2. Bucket virtual-hosted -Style URL endpoint.

Example: https:// bucket-name. s3- ap-northeast-1.amazonaws.com.

-> We can access the Amazon S3 through:

1. Aws management console

2. Aws Command Line interface

3. SDK.

Applications

1. Backup and storage

2. Application hosting

3. Media hosting

4. Software delivery.

-> We have to pay only for what we use in addition to PUT, COPY, POST, LIST, and GET requests that transfer out to other regions in GB per month

-> Once the bucket is created, it is automatically associated with specific AWS regions.

Amazon EFS

-> Amazon EFS Implement Storage for Ec2 instances that multiple virtual machines can access at the same time across multiple Availability Zones, Vpc, and regions

-> It is implemented as a shared file system that uses the network file system(File storage) and it has many features:

1. Petabyte-scale, low-latency file system

2. Shared Storage and elastic capacity.

3. Supports network file systems (NFS) versions 4.0 and 4.1

4. Compatible with all Linux-based AMI’s for amazon EC2.

Amazon EFS Implementation

-> create your amazon EC2 resources and launch your Amazon Ec2 instances.

-> Create your Amazon EFS file system.

-> Create your mount targets in the appropriate subnets connect your Amazon ec2 instances to mount targets and then verify the resources and protection of your Aws account.

-> Transfer acceleration is the ability provided by Aws to enable, fast, easy, and secure transfer of files over long distances between your client and Amazon S3 bucked.

-> Amazon S3 versioning is the best way to protect your data from accidental deletion on Amazon S3

Amazon S3 Glacier

-> It is a Secure, durable, and low-cost cloud storage service for data archiving and long-term backups.

-> It is also designed to provide 11 9’s of durability for objects.

-> It Supports the encryption of data in transit and at rest through Secure Socket Layers (SSL) or Transport Layer Security (TLS).

-> The Vault Lock feature enforces compliance through a policy and it has an extremely low-cost design that works well for long team archiving. (retrieval times ranging from minutes to hours) and is used to store data that is not frequently accessed.

Use Cases

1. Media asset archiving

2. Healthcare information archiving

3. Regulatory and compliance archiving

4. Scientific data archiving

5. Digital preservation and magnetic tape replacement

-> we use amazon S3 for lifecycle policies not amazon S3 glacier

-> using and accessing the Amazon S3 Glacier through:

1. RESTFUL Web Services

2. Java or .NET SDKS

3. Amazon S3 with lifecycle policies, Aws glacier Aps, Aws and glacier SDK

-> Amazon S3 lifecycle policies, enable you to delete or move objects based on age;

-> It has many storage classes (mainly of S3) like:

1. S3 standard

2. S3 Standard — Infrequent Access

3. S3 Intelligent — tiering

4. S3 one zone — Infrequent Access

5. S3 Glacier

6. S3 Glacier Deep Archive

-> The name of an S3 bucket must be unique worldwide across all AWS accounts

-> All data stored in S3 is not viewable by the public and the vault in Amazon S3 Glacier is a container for storing archives and S3 also offers virtually unlimited storage.

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Congratulations! You’ve unlocked the secrets of efficient and scalable storage solutions within the AWS cloud. This knowledge empowers you to choose the right storage options for diverse data needs and optimize your cloud storage costs.

Leave a comment below with any questions or cloud computing concepts you’d like to explore further!

In the next module, Module 8: Databases, we’ll embark on a journey to explore the wide range of managed database services offered by AWS. Get ready to discover solutions for various database workloads and leverage the power of cloud-based databases!

Episode 6: https://medium.com/@harshithavineni81/mastering-the-cloud-module-6-compute-f8d5e66484bf

Episode 8: https://medium.com/@harshithavineni81/mastering-the-cloud-module-8-databases-7ec1c43f6fde