Amazon S3 Storage Classes
Introduction
Amazon Simple Storage Service (S3 ) as the name indicates it is simple object storage built for storing and retrieving data from anywhere on the internet. Whatever the data you upload to it, all are considered as objects. Objects can be stored in the form of photos, videos, audios, a large number of binaries and other object forms.
The total volume of data and the number of objects that can be stored are unlimited. An object is similar to a file it has a key(name) and value(data). Individual objects can contain 0 to 5TB
Amazon S3 provides different types of storage classes for the data we stored on S3 in the form of objects. This helps to choose the most suitable storage in terms of accessibility and storage period. Here we discuss every storage class/tier along with lifecycle management.
Mostly the data we are storing is divided based on their access and storage patterns. That is how frequently we are accessing the data or not.
For frequently accessed data, the storage classes are S3 Standard and Reduced Redundancy. This can be accessed within milliseconds.
S3 Standard
This is the default storage class of S3, so even though you don’t specify the storage class it is allocated automatically. The data is replicated in more than 3 AZs. This doesn’t need any minimum object size, min storage duration, and no retrieval fee. It gives low latency so that content is delivered so fastly and provides high throughput. It is designed for 99.9% availability over a year. It is designed to get 99.999999999% durability of objects across multiple AZs.
Reduced Redundancy
This storage class is used for non-sensitive data in the sense the data is non-critical and can be reproduced. This can also replicate more than 3 AZs, no minimum storage duration and object size is needed and no retrieval fee. AWS does not recommend to use this storage, because these objects have an average annual expected loss of 0.01 percent of objects. If the object of this is lost and tried to access that object it will through 405 error.
For infrequent access data Standard-IA and One zone -IA. These are also the same as S3 Standard in terms of availability, objects can be accessed in milliseconds.
Standard Infrequent Access ( Standard IA)
Standard Infrequent can be used for the objects where realtime access is required but infrequent. The data can be replicated to more than 3 AZs. This is cheaper than Standard because of infrequent access. Resilient against events will affect the entire Availability Zone.
One Zone Infrequent Access ( One Zone -IA)
This can be used for non-critical and reproducible objects. As the name indicates the data is stored only in one zone. This effect costs too, it is 20% costs less than Standard -IA. If this AZ is failed then the whole data will be lost. This is more useful when data need to replicate using S3 Cross-Region Replication. It is useful to store the secondary backup copies of data or easily re-creatable.
Both these storage classes are suitable when you have a minimum of 128 KB data and a minimum of 30 days storage period. Both charges for retrieving data.
AWS provides storage classes for low cost archiving data. These are S3 Glacier and S3 Glacier Deep Archive.
S3 Glacier
This is useful when we need long-term archival storage ( like warm or cold backups). Data stored in this has a minimum storage period of 90 days. A retrieval could take minutes or hours. Faster the data retrieval higher the cost. The data will be replicated in more than 3 AZs. The minimum billed object size is 40 KB. Even though you moved data to another storage class or data is deleted still, you are charged for 90 days. You can directly upload files to this using PUT API and lifecycle managements for automatic migrations objects. Resilient against events will affect the entire Availability Zone.
S3 Glacier Deep Archive
This is useful when you need long time backups that need low access to them. It uses 180 days of minimum storage duration. It doesn’t matter what you do with your data means whether you delete, overwrite or move to another storage class you are still charged for 180 days. Even though you used it for one day it will charge for 180 days. This is a replacement for tap -style storage. This is the cheapest storage of all S3 storage classes. You can still reduce this cost by bulk retrieval which returns the data within 48 hours. The minimum billed object size is 40 KB. You can directly upload files to this using PUT API and lifecycle managements for automatic migrations objects. Resilient against events will affect the entire Availability Zone. It is designed to get 99.999999999% durability of objects across multiple AZs.
Objects that we stored in this Glacier and Glacier Deep Archive are not real-time accessible. To access objects first, we have to put a request and it will return a temporary copy of the data with available duration. The available duration is sent along with the request.
Intelligent Tiering
The name itself indicates that it is intelligent among all storage types. It is a special type of storage class that optimizes costs by automatically moving data to the most cost-effective tiers. In these operations ongoing in there will be no difference in performance impacts. It moves objects automatically between two tiers — one designed for frequent access, other for infrequent access. The Intelligent Tiering is the best solution when the access patterns are unknown or unpredictable of long-living objects and you want it to be most cost-optimized.
Intelligent Tiering stores data in two storage classes, where one is optimized for frequent access and another for infrequent access. For achieving this S3 monitor the objected access patterns when an object infrequent storage class is not accessed in consequent 30 days then that object will be moved to infrequent storage which offers a lower cost.
If an object in infrequent storage class is accessed then automatically the object will be moved back to the frequent access storage. There is no charge for moving objects between access tiers.
It adds monthly charges for monitoring and automation. There is no fee for retrieving objects. The minimum size of the objects is 120KB. The minimum storage duration is 30 days. If the object is less than 120 KB you are still charged for 120KB, so it is not suitable for the objects which are below 120KBs. If you delete or overwrite an object before the minimum storage period of 30 days, you are still charged for 30days. So when objects storage is less than 30days it is not the best solution. Instead, you can use the Standard storage tier. It is the same as Standard tier in terms of low latency, throughput, availability, and durability.
Object Lifecycle Management
For most cost optimization S3 provides lifecycle methods for an object. We can use this in different scenarios like when we are storing logs of an application for monitoring for a particular period and after that, they may delete or archive, we need to store the data according to the current trend and after that trend, we don’t need them anymore.
Lifecycle Rules
There are certain lifecycle rules for the objects stored in S3 and they are used to control them. This allows us to automate the transition of objects between storage tiers and in some cases, if we do not need them for a longer duration then they can be deleted. The life cycle rules are added at the bucket level. At any time we can enable and disable according to our needs and requirements.
Objects smaller than 120KB cannot be moved to Intelligent tiring. The objects must be in original storage for 30 days, then only it can be moved to another storage class based on their access patterns. We can configure objects expiring time after certain periods. At the point of the object expiry, they are deleted from the bucket.
Objects can be archived into Glacier using this lifecycle configuration. The objects remain inside S3 and managed by S3 but they are stored in Glacier. Objects can be restored from the Glacier by requesting it. It will return data for temporary periods after that they are deleted. If objected are encrypted, they remain encrypted in glacier and temporary restoration into S3.
Conclusion :
For different types of business needs, we need different types of storage types that support them. If you are clear about your data storage you can use any specific one from mentioned above. If you are unclear about storage use Intelligent tier and life cycle management.
