AWS Storage Gateway
A Comprehensive Guide to become a Certified Solutions Architect Associate
Storage gateway is a hybrid cloud storage solution that connects an on-premise software appliance (with cloud storage) to the AWS cloud. This allows you to integrate your on-premise applications and workflows with AWS storage (block and object storage services) through secure industry standard protocols.
It will store files as S3 objects, archive virtual tapes (Glacier), and stores EBS snapshots (via Volume Gateway/EBS). It’s actually available for download as a VM (virtual machine) image which you install on a host within your datacenter. The acts as a portal or ‘gateway’ to the AWS cloud. It supports VMware ESXi or Microsoft Hyper-V as hypervisors. A hypervisor basically runs virtual machines.
Storage gateways offers three different types or solutions:
- File Gateway (NFS/SMB)
- Volume Gateway (iSCSI)
- Cached Volumes
- Stored Volumes
- Tape Gateway (Virtual Tape Library)
File Gateway
How it works is your files and metadata will be stored as objects in your S3 buckets, these can be accessed through a NFS mount point (NFS v3.0 or v4.1). If you’re using SMB, version 2 and 3. Once you’ve transferred your files (objects) to S3, your bucket policies, lifecycle management, versioning, and cross-region replication will apply to all objects in your buckets.
File gateway currently supports:
- S3
- S3 — IA
- S3 — One Zone — IA
Volume Gateway
The volume gateway is the iSCSI block protocol solution that presents your applications with disk volumes. The data written inside of these volumes can be asynchronously backed up as a point-in-time snapshot and also stored in the cloud as EBS snapshots. A snapshot is an incremental backup meaning it only captures changed blocks (modifications). The snapshot storage is all compressed.
Cached Volumes
Allows you to store primary data locally for frequently accessed data volumes (hence ‘cached’).
- Each gateway can handle up to 32 cached volumes (configured as cached volumes).
- These volumes can range in size from 1 TiB — 32 TiB (rounded to nearest TiB).
Stored Volumes
Stored volumes are the way to go for access to low-latency datasets. Your primary data is stored locally while asynchronously being backed up to AWS. To enable this, you would configure your on-prem gateway to store all data locally then back up snapshots of this data to S3.
- These range in size from 1 TiB — 16 TiB (rounded to nearest GiB).
- Each gateway can handle up to 32 stored volumes.
Tape Gateway
Allows you to store your archival data on virtual tape cartridges in Amazon Glacier. The virtual tape library (VTL) interface allows you to leverage your existing tape backup application to store data on virtual tapes in the tape gateway. You would deploy your gateway on an EC2 instance to provision iSCSI storage volumes in AWS.
- Supported by NetBackup, Backup Exec, Veeam, etc.
- If you have a tape in Glacier, you can retrieve it in 3–5 hours. If it’s in Deep Glacier, it will take about 12 hours.
Hardware Appliance
Storage gateway is also available as a physical hardware appliance (yes, it even comes with Amazon Prime). This is a Dell EMC PowerEdge R640 server with Storage Gateway pre-installed on it with a validated configuration. You manage the appliance from the AWS management console or API.
You would typically use the physical appliance to simplify deploying, procuring, and managing Storage Gateway for on-prem IT invironments like remote offices or branches lacking:
- An existing virtual server infrastructure
- Adequate disk/memory resources
- Staff with hypervisor management skills
This enables you to avoid procuring additional infrastructure needed for a virtual environment to operate the local Storage Gateway VM appliance.
Do be aware that this appliance only allows you to run one gateway at a time. However, you can change the type of gateway once installed on the hardware appliance. You can add more storage (5 x 1.92 TB SSD’s) and it does support software-based ZFS RAID. This provides protection for the base model against 1 SSD failure (5 TB of usable storage) and 2 SSD failure (12 TB usable storage).
For more information on Storage Gateway, you can read the FAQ or AWS Documentation.
