Creating AWS File Gateway as an NFS storage for your EC2 instance.
This Blog has moved from Medium to blogs.tensult.com. All the latest content will be available there. Subscribe to our newsletter to stay updated.
NAS or Network Attached Storage is a device connected to a network that allows the storage and retrieval of data from a central location from multiple users and clients. NAS storages are fast, flexible, easy to scale and cheap. AWS has a service called AWS EFS which can be used for accessing files from multiple EC2 instances. To know more about EFS visit the documentation here.
Shared storage for multiple EC2 instances can be easily achieved using AWS EFS, but unfortunately, it was not available in Mumbai region at the time of writing this blog. So I thought of an alternative to avoid using the costly data management solutions available in the market.
Edit: AWS EFS is now available in the Mumbai region.
In AWS own words, AWS Storage Gateway connects an on-premises software appliance with cloud-based storage to provide seamless integration with data security features between your on-premises IT environment and the AWS storage infrastructure. You can use the service to store data in the AWS Cloud for scalable and cost-effective storage that helps maintain data security.
Here in this blog, we will be setting up a File Gateway using the NFS protocol for accessing the storage from multiple EC2 instances. To know more about AWS Storage Gateway, visit the documentation here.
What is a File Gateway?
A file gateway helps in integrating the existing application to the S3 storage using industry-standard file protocols such as NFS (Network File System) and SMB (Server Message Block). File gateway will cache the frequently used files in an EBS cache volume attached to the gateway and writes the file into an S3 bucket for storage. The gateway can be deployed into your on-premises environment as a virtual machine (VM) running on VMware ESXi or Microsoft Hyper-V hypervisor or it can be launched on an EC2 instance using the custom gateway AMI from AWS.
The cache storage files will be removed or overwritten only if the storage space is low and a new file needs to be added to the cache. With that being said, maintaining a bigger cache volume can avoid the caching of object frequently from the S3 bucket against new requests.
Now let us look into the steps for setting up file gateway as a NAS,
- Login to your AWS console and navigate to the storage gateway service. You will be able to see the below screen. Click on the “Get started” to proceed.
2. In the given options, select the gateway type as file gateway and click next to continue.
3. In this example we will be using an EC2 instance as the gateway, so in the next step select the host platform as “Amazon EC2”. To continue to the next step we need to create the gateway VM first.
4. As mentioned in the previous step, click on the Launch Instance and you will be redirected to the EC2 creations page. The instructions for creating the VM are already available, please refer to the documentation here.
The instance should be able to access the internet, in this case I will be launching the instance as public but this can also be achieved by a NAT gateway for private instances.
5. The recommended configuration for gateway VM is m4.xlarge, for this example I will be choosing an instance from t2 family. When creating the instance make sure that the following ports are opened in the security group for NFS access. Port 80 is used for activation and this can be removed once the gateway is successfully configured. Source IP’s for the NFS ports should be the instance IP that needs to be connected with the storage gateway.
Also, the minimum storage for cache volume is 150GiB of EBS volume.
6. Once the gateway instance is up and running, click next and enter the IP address of the gateway instance and connect. Make sure that the instance is accessible from your browser otherwise this step will not proceed further.
7. Proceed to activate the gateway, change the time zone settings to your preference and name the gateway. Activation requires Internet access, if any other publicly accessible instances are available that can be also used for activation.
8. Once the gateway is successfully activated, configure the attached EBS volume as a cache and save the changes.
9. Now we have successfully configured the gateway. If all the steps are done correctly you can see the gateway status as running. Proceed to create a file share to store our files, click on “Create file share”.
10. For creating the file share you need an S3 bucket. Enter the name of the S3 bucket, select NFS as the protocol and the gateway we created will be available under the gateway section. Click next to continue.
11. Depending on your use case select the S3 storage options. Here in this example I will be using the Standard tier. Also, check the option “Create a new IAM role” as this will automatically create a role for S3 access.
12. Now review the configuration and create the file share. Once the configuration is completed you will get the mount point that can be connected to the EC2 instance.
13. Log on to the instance and connect to the file share using the commands. In this example, I will be using a Linux instance to connect to the file share. Once the instance is connected with the file storage you can start putting the files inside the cache volume and this files will be replicated to the S3 bucket. Similarly, other instances can also be connected to the NFS storage using the same mount point.
Besides, if you are planning to launch a private instance for the gateway following step can be followed for activating the file gateway,
- Log on to a Bastion Host instance and try hitting the private IP of the gateway using the following command, here I am using the region ap-south-1 where my gateway is launched.
wget "http://<gateway-instance-ip>/?gatewayType=FILE_S3&activationRegion=ap-south-1"Once you enter the command, it will publish a temporary link which can be hit using your browser to activate the link as given below,
Feel free to test it out and let us know your feedback.
