Block and File Storage

Amazon EBS

• EBS volumes are data persistent
  • EBS volumes do not need to be attached to an instance
  • You can attach multiple EBS volumes to an instance
  • You can use multi-attach to attach a volume to multiple instances but with some constraints
  • EBS volumes must be in the same AZ as the instances they are attached to
  • Root EBS volumes are deleted on termination by default
  • Extra non-boot volumes are not deleted on termination by default

Amazon EBS HDD-Backed Volumes

Amazon Data Lifecycle Manager (DLM)

• DLM automates the creation, retention, and deletion of EBS snapshots and EBS-backed AMIs
• DLM helps with the following:
  • Protects valuable data by enforcing a regular backup
  schedule
  • Create standardized AMIs that can be refreshed at regular
  intervals
  • Retain backups as required by auditors or internal
  compliance
  • Reduce storage costs by deleting outdated backups
  • Create disaster recovery backup policies that back up data to isolated accounts

EBS vs instance store

• Instance store volumes are high performance local disks that are physically attached to the host computer on which an EC2 instance runs
  • Instance stores are ephemeral which means the data is lost when powered off (non-persistent)
  • Instance stores are ideal for temporary storage of information that changes frequently, such as buffers, caches, or scratch data
  • Instance store volume root devices are created from AMI templates stored on S3
  • Instance store volumes cannot be detached/reattached

Amazon Machine Images (AMIs)

• An Amazon Machine Image (AMI) provides the information required to launch an instance
• An AMI includes the following:
  • One or more EBS snapshots, or, for instance-store-backed AMIs, a template for the root volume of the instance (for example, an operating system, an application server, and applications)
  • Launch permissions that control which AWS accounts can use the AMI to launch instances
  • A block device mapping that specifies the volumes to attach to the instance when it’s launched
• AMIs come in three main categories:
  • Community AMIs — free to use, generally you just select the operating system you want
  • AWS Marketplace AMIs — pay to use, generally come packaged with additional, licensed software
  • My AMIs — AMIs that you create yourself

EBS Snapshots

• Snapshots capture a point-in-time state of an instance
  • Cost-effective and easy backup strategy
  • Can be used to migrate a system to a new AZ or region
  • Can be used to convert an unencrypted volume to an encrypted volume
  • Snapshots are stored on Amazon S3
  • EBS volumes are AZ specific but snapshots are region specific

Amazon Elastic File System (EFS)

• Fully-managed file system solution
  • Accessed using the NFS protocol
  • Elastic storage capacity and pay for what you use
  • Multi-AZ metadata and data storage
  • Can configure mount-points in one, or many, AZs
  • Can be mounted from on-premises systems ONLY if using Direct Connect or a VPN connection
  • Alternatively, use the AWS DataSync
  • EFS is elastic and grows and shrinks as you add and remove data
  • Can scale up to petabytes
• Can concurrently connect up to 1000s of EC2 instances, from multiple AZs
  • Can choose General Purpose or Max I/O (both SSD)
  • Data is stored across multiple AZ’s within a region
  • Read after write consistency
  • Need to create mount targets and choose AZ’s to include

EFS Access Control

• You can control who can administer your file system using IAM
• You can control access to files and directories with POSIX-compliant user and group-level permissions
• POSIX permissions allow you to restrict access from hosts by user and group
• EFS Security Groups act as a firewall, and the rules you add define the traffic flow

EFS Encryption

• EFS offers the ability to encrypt data at rest and in transit
• Encryption at rest MUST be enabled at file system creation time
• Encryption keys are managed by AWS KMS
• Data encryption in transit uses industry standard Transport Layer Security (TLS)

AWS DataSync

• Provides a fast and simple way to securely sync
existing file systems into Amazon EFS
• Securely and efficiently copies files over the internet
or an AWS Direct Connect connection
• Copies file data and file system metadata such as
ownership, timestamps, and access permissions

Amazon FSx

• Amazon FSx provides fully managed thirdparty
  file systems
• Amazon FSx provides you with two file systems to choose from:
  • Amazon FSx for Windows File Server for
  Windows-based applications
  • Amazon FSx for Lustre for compute-intensive
  workloads

Amazon FSx for Windows File Server

• Provides a fully managed native Microsoft Windows
  file system
• Full support for the SMB protocol, Windows NTFS, and Microsoft Active Directory (AD) integration
• Supports Windows-native file system features:
  • Access Control Lists (ACLs), shadow copies, and user
  quotas.
  • NTFS file systems that can be accessed from up to
  thousands of compute instances using the SMB protocol
• High availability: replicates data within an Availability Zone (AZ)
• Multi-AZ: file systems include an active and standby file server in separate AZs

Amazon FSx for Lustre

• High-performance file system optimized for fast processing of workloads such as:
  • Machine learning
  • High performance computing (HPC)
  • Video processing
  • Financial modeling
  • Electronic design automation (EDA)
• Works natively with S3, letting you transparently access
  your S3 objects as files
• Your S3 objects are presented as files in your file
  system, and you can write your results back to S3
• Provides a POSIX-compliant file system interface

AWS Storage Gateway — File Gateway

• File gateway provides a virtual on-premises file server
• Store and retrieve files as objects in Amazon S3
• Use with on-premises applications, and EC2-based applications that need file storage in S3 for objectbased workloads
• File gateway offers SMB or NFS-based access to data in Amazon S3 with local caching

AWS Storage Gateway — Volume Gateway

• The volume gateway supports block-based volumes
• Block storage — iSCSI protocol
• Cached Volume mode — the entire dataset is stored on S3 and a cache of the most frequently accessed data is cached on-site
• Stored Volume mode — the entire dataset is stored on-site and is asynchronously backed up to S3 (EBS point-in-time snapshots). Snapshots are incremental and compressed

AWS Storage Gateway — Tape Gateway

• Used for backup with popular backup software
• Each gateway is preconfigured with a media changer and tape drives. Supported by NetBackup, Backup Exec, Veeam etc.
• When creating virtual tapes, you select one of the following sizes: 100 GB, 200 GB, 400 GB, 800 GB, 1.5 TB, and 2.5 TB
• A tape gateway can have up to 1,500 virtual tapes with a maximum aggregate capacity of 1 PB
• All data transferred between the gateway and AWS storage is encrypted using SSL
• All data stored by tape gateway in S3 is encrypted server-side with Amazon S3-Managed Encryption Keys (SSE-S3)