AWS Transfer Family For File Transfers
Have you ever wondered about hosting, managing SFTP servers and provisioning seamless channels between the teams or third party vendors to transfer files or data? Your search ends here! AWS transfer family, a managed service from AWS will help you to achieve it.
Introduction
AWS Transfer Family is a fully managed AWS service that could be leveraged to transfer files in and out of Amazon S3 or Amazon EFS over the following protocols:
- Secure Shell (SSH) File Transfer Protocol (SFTP)
- File Transfer Protocol Secure (FTPS)
- File Transfer Protocol (FTP)
AWS Transfer Family supports up to 3 Availability Zones and is backed by Auto scaling with redundant fleet for connections and transfer requests. Users would be served by creating a file transfer protocol-enabled server in the AWS transfer family, followed by creating user-access.
Prerequisites
- An AWS account
- Administrative privileges or read/write access to AWS Transfer family, IAM,S3,EC2,AWS Directory service and CloudWatch
Architectural Design
We would consider below use case for demonstration i.e. AWS transfer family with AWS managed MS Active Directory.
Implementation
Step1: Login Into AWS Account
Step2: Create a private S3 bucket, I have named it as “s3-sftp-transferfamily” consisting of folders for different partners or teams, that we would require later.
Step3: Create an IAM role with permissions to access S3 bucket created in earlier step and any objects within it. In my example its named “Role-s3-sftp-transferfamily”.
Step4: Create AWS managed Microsoft Active directory. Provision groups and users in the directory as shown below.
Groups “AWS Transfer family” and “AWS Transfer family 2” have been created and users AWS Bot1, AWSBot3 added in former and AWSBot2 added in the latter.
Step5: Create a SFTP server endpoint with a default DNS name in AWS transfer family service.
Step6: Create access for AD users within AWS Transfer family associating IAM role created in step3 and assigning home directory as S3 bucket. AD group SID is required while creating access for an AD group.
NOTE — Use the following Windows PowerShell command to retrieve the SID for a group, replacing YourGroupName with the name of the group.
Get-ADGroup -Filter {samAccountName -like “YourGroupName*”} -Properties * | Select SamAccountName,ObjectSid
Newly created access in AWS Transfer family for the users resides in AD groups such as AWS Transfer family and AWS Transfer family 2 according to their SIDs as shown below.
Step7: Grant access to the cloudwatch service using newly created role “AWSTransferLoggingAccess” to manage and store logs into cloudwatch log-group tracking all the request made SFTP server endpoint.
Step8: Access the server using FTP client such as WinSCP. Enter Server-endpoint details and AD user-credentials. Once you click on the Login button, you will be navigated to the user assigned home directory in the S3 bucket.
Step9: Create test files, I have used “bot1.txt” and “Demo1 SFTP.txt”, drag and drop from your local system folder into the SFTP server directory.
File uploaded earlier will automatically reflect into S3 bucket as shown below for “MS AD bot1” directory, transfer is successful..!!!
Summary
We have demonstrated the file transfer via SFTP protocol leveraging AWS transfer family and AWS managed Microsoft Active directory with different users onto a SFTP server endpoint with restricted access. A monitoring mechanism leveraging Cloudwatch has also been deployed. The AWS transfer family would be a decent choice to do away with infrastructure management overhead and ensure cost and time optimization.
References
AWS Transfer Family https://docs.aws.amazon.com/transfer/latest/userguide/what-is-aws-transfer-family.html
