AWS: How to Mount S3 Bucket on EC2 Linux Instance Using IAM Role

This blog has been moved from medium to blogs.tensult.com . All the latest content will be available there. Subscribe to our newsletter to stay updated.

We can mount an S3 bucket onto an AWS instance as a file system known as S3fs. It is a FUSE filesystem application backed by amazon web services, that allows you to mount an Amazon S3 bucket as a local file-system. We can use system commands with this drive just like as any other Hard Disk in the system. On s3fs mounted files systems we can simply use cp, mv and ls the basic Unix commands similar to run on locally attached disks.

Filesystem in Userspace (FUSE) is a software interface for Unix and Unix-like computer operating systems that lets non-privileged users create their own file systems without editing kernel code. This is achieved by running the file system code in user space while the FUSE module provides only a “bridge” to the actual kernel interfaces.

Why S3 Bucket?

We can consider NFS sort of solution, even now we have EFS from Amazon but it’s costly and even the same data were used for their analytics solution. So we thought to use S3 to satisfy both the requirement.

Follow the below steps to mount your S3 bucket to Your Linux Instance.

We are assuming that you have a running Linux EC2(Red Hat/Centos) instance on AWS with root access and a bucket created in S3 which is to be mounted on your Linux Instance.

Step-1: Using new instance of CentOS or Red Hat.Update the system.

#sudo yum update

Step-2: Install Required Packages

First, we will install all the dependencies for fuse and s3cmd. Install the required packages to system use following command.

# sudo yum install automake fuse fuse-devel gcc-c++ git libcurl-devel libxml2-devel make openssl-devel

Step-3: Download s3fs source code from git.

# git clone https://github.com/s3fs-fuse/s3fs-fuse.git

Step-4 :Now Compile and install the code.

Following the set of command will compile fuse and add fuse module in the kernel.

# cd  s3fs-fuse
# ./autogen.sh 
# ./configure — prefix=/usr — with-openssl
# make 
# sudo make install

Step-5: Use below command to check where s3fs command is placed in os.

# which s3fs

Step-6: Creating a IAM role for s3 bucket

Create one IAM role with policy having appropriate access to particular bucket.

For example :- My IAM role name is s3fsmountingrole and bucket created is s3fs-demobucket

Policy attached should be read/ write access for bucket s3fs-demobucket

Enter policy name Description and Policy Document as given below

{
 “Version”: “2012–10–17”,
 “Statement”: [
 {
 “Effect”: “Allow”,
 “Action”: [
 “s3:GetBucketLocation”,
 “s3:ListAllMyBuckets”
 ],
 “Resource”: “arn:aws:s3:::*”
 },
 {
 “Effect”: “Allow”,
 “Action”: [“s3:ListBucket”],
 “Resource”: [“arn:aws:s3:::s3fs-demobucket”]
 },
 {
 “Effect”: “Allow”,
 “Action”: [
 “s3:PutObject”,
 “s3:GetObject”,
 “s3:DeleteObject”
 ],
 “Resource”: [“arn:aws:s3:::s3fs-demobucket/*”]
 }
 ]
}

Attach IAM Role to the running Instance or Launching new Instance

Step-7: Now create a directory or provide the path of an existing directory and mount S3bucket in it.

#sudo mkdir -p /var/s3fs-demofs

Step-8: Now mount the s3 bucket using IAM role enter following command :

#s3fs -o iam_role=”s3fsmountingrole” -o url=”https://s3-eu-central-1.amazonaws.com" -o endpoint=eu-central-1 -o dbglevel=info -o curldbg -o allow_other -o use_cache=/tmp s3fs-demobucket/var/s3fs-demofs

Step-9: Check mounted s3 bucket. The output will be similar as shown below but Used size may differ.

#df -h

df -h shows the mounted file system, here you can see we have successfully mounted the S3 bucket on your EC2 Instance.

Note: If you already had some data in s3bucket and it is not visible, then you have to set permission in ACL at the S3 AWS management console for that s3 bucket.

Congrats!! You have successfully mounted your S3 bucket to your EC2 instance.

Conclusion

Here, I explained how to mount AWS s3 bucket on EC2 Linux instance, and for demo purpose, I used RedHat machine and created one IAM role for access to s3 bucket and attached it to running instance. You can also get access to s3 bucket from EC2 instance by providing AWS access key and secret key.