AWS S3 : SIMPLE STORAGE SERVICE

What is AWS S3 ?

Amazon Simple Storage Service (S3) is a storage for the internet. It is designed for large-capacity, low-cost storage provision across multiple geographical regions. Amazon S3 provides developers and IT teams with Secure, Durable and Highly Scalable object storage.

S3 is Secure because AWS provides:

• Encryption to the data that you store. It can happen in two ways:
• Client Side Encryption
• Server Side Encryption
• Multiple copies are maintained to enable regeneration of data in case of data corruption
• Versioning, wherein each edit is archived for a potential retrieval.

S3 is Durable because:

• It regularly verifies the integrity of data stored using checksums e.g. if S3 detects there is any corruption in data, it is immediately repaired with the help of replicated data.
• Even while storing or retrieving data, it checks incoming network traffic for any corrupted data packets.

What kind and how much of data one can store in AWS S3?

You can store virtually any kind of data, in any format, in S3 and when we talk about capacity, the volume and the number of objects that we can store in S3 are unlimited.

*An object is the fundamental entity in S3. It consists of data, key and metadata.

When we talk about data, it can be of two types-

• Data which is to be accessed frequently.
• Data which is accessed not that frequently.

How is data organized in S3?

Data in S3 is organized in the form of buckets.

• A Bucket is a logical unit of storage in S3.
• A Bucket contains objects which contain the data and metadata.

Before adding any data in S3 the user has to create a bucket which will be used to store objects.

How is the data transferred?

Besides traditional transfer practices that is over the internet, AWS has 2 more ways to provide data transfer securely and at a faster rate:

• Transfer Acceleration
• Snowball

Transfer Acceleration enables fast, easy and secure transfers over long distances by exploiting Amazon’s CloudFront edge technology.

CloudFront is a caching service by AWS, in which the data from client site gets transferred to the nearest edge location and from there the data is routed to your AWS S3 bucket over an optimised network path.

Amazon S3 Storage Types

With the largest global cloud infrastructure today, Amazon has built Amazon S3, a highly durable and scalable solution which provides multiple storage optionsdesigned to fit specific customer needs. These include:

• Standard: Used to store performance-sensitive data that should have a retrieval time of milliseconds.
• Standard Infrequent Access: Used to store infrequently accessed data.
• One Zone-Infrequent Access: Used for infrequently used objects that need lower durability. Saves cost compared to other storage types.
• Amazon Glacier: Used to store archived data.

Amazon S3 Standard StorageAmazon S3 Standard Infrequent AccessAmazon S3 One Zone-Infrequent AccessAmazon GlacierDurability99.999999999%99.999999999%99.999999999%99.999999999%Availability99.99%99.9%99.5%NASLA99.9%99%99%NAObject storage fee (first 50TB)$0.023$0.0125$0.01$0.004Object retrieval feeNA$0.01 per GB$0.01 per GBPer GB based on retrieval rateNumber of Availability Zones where objects are stored>= 3>= 31>= 3Retrieval timeMillisecondsMillisecondsMillisecondsMinutes or hours

Amazon S3 Use Cases

Amazon S3 has many use cases, including:

Storage for Internet

Amazon S3 is ideal when you want to store application images and videos, and render with faster performance. All AWS services (including Amazon Prime and Amazon.com), as well as Netflix and Airbnb, use Amazon S3 for this purpose. Combining Amazon S3 with Amazon CloudFront enables much faster delivery due to CloudFront’s edge locations.

Backup and Disaster Recovery

Amazon S3 is suitable for storing and archiving highly critical data or backup because it is automatically replicated cross-region, providing maximum availability and durability. For even more protection, you can use Amazon S3 versioning, which stores multiple versions of each file so it’s easy to recover the files or older copies. With Amazon S3, it’s rare to lose data if you keep your recovery point objective (RPO) and recovery time objective (RTO) as low as possible.

Analytics

Amazon S3 provides a sophisticated in-place querying functionality to run powerful analytics on data which is in rest on S3. It eliminates the need to move and store data, as it supports a majority of third-party service integrations.

Data Archiving

You can store and move TBs of data from Amazon S3 to Amazon Glacier’s very cheap and durable archiving solution for compliance purposes. You can also automate when data should be archived with a lifecycle policy that helps reduce efforts to manage data.

Static Website Hosting

Amazon S3 stores various static objects. One interesting use case is its ability to host static websites. More and more web apps are becoming single page and static (Angular, ReactJS, etc.), and it’s costly to keep running a web server for their hosting. S3 offers a static website hosting feature that will enable you to use your own domain without incurring huge web server hosting costs.

Security and Compliance

Amazon S3 provides multiple encryption and compliance standard features for PCI-DSS, HIPAA/HITECH, FedRAMP, the Data Protection Directive, FISMA, and more. These features help customers satisfy compliance requirements for virtually every regulatory agency around the world. They also make it easy to limit access access to critical data with the help of bucket policies.

Getting Started with Amazon S3

Step 1: Create an S3 Bucket

First, create a bucket by logging in to AWS Management Console or via AWS Command Line Interface (AWS CLI). By default, you can create up to 100 buckets in an account, but this soft limit can be extended with a request.

Go to the Amazon S3 console and click “Create bucket.”

Then choose a unique bucket name, according to the Amazon S3 bucket namingrules, and click “Create.” You can set up the configuration and permission of a bucket in the same window, but that can be enabled or changed later according to need.

Step 2: Configure Options (Optional)

In “Configure options,” you can select features you want to enable on a particular bucket, such as:

• Versioning: Keeps track of all versions of a file, making it easy to recover the file in case of accidental deletion.
• Server access logging: Logs all requests/activities on your bucket to another bucket.
• Tags: You can tag the bucket with key and name, which will make it easier to search resources with tags.
• Object-level logging: Enable this feature if you want to record every activity for every object in the bucket.
• Default encryption: By default, AWS encrypts files with AES 256, but you can use your own managed key to encrypt objects.

Step 3: Set Permissions (Optional)

There are various ways to grant permission to Amazon S3 buckets. By default, permission is private, but this can be changed using the AWS Management Console permission or bucket policy. It is best to keep the default permissions.

As a security best practice, you should be selective when granting access to Amazon S3 buckets. Only add permissions which are necessary and avoid keeping buckets open to the public.

Step 4: Add Permissions Using an S3 Bucket Policy

When you set your bucket policy, you can grant users various granular level permissions on different actions. You can create a permission/policy using the AWS Policy Generator.

In the below policy, all objects in the bucket are publicly accessible to anyone viewing over the Internet. However, the policy only allows public view permission to all objects. Users viewing over the Internet cannot modify or update objects, thus restricting users’ permission to the “only view” category. You can find bucket policies like these here.