Mounting your Object Storage Bucket as a File System on your ECS Instance

Sylia CH
Sylia CH
Jun 24, 2020 · 4 min read
Photos via Pexels

Introduction

In this article , i’ll be dealing with Flexible Engine Cloud Provider in order to mount my Object Storage Bucket as a File System on my ECS Instance.

Establishing communication between our ECS instances and the Object Storage Bucket is crucial for facilitating different daily exchanges such as moving our local files and directories directly to the mounted Bucket without having to use the Cloud Provider’s Object Storage Console.

This process can be done in two main ways :

In this tutorial, i’ll focus on the s3fs-fuse project as it is a lot more easier to access Object Storage Bucket content directly using a file system then through scripts. However, if you’re interested on how to deal with Flexible Engine Python SDK to access your Object Storage Bucket, you can checkup my next article.

Prerequisites:

In order to get started, all what you need is a Flexible Engine account (or any Cloud Provider account that provides an Object Storage API compatible with the s3fs-fuse project), a Linux compute instance and of course an SSH access.

You also need to connect to your Cloud Provider account and setup up an IAM policy for your Object Storage Bucket :

  • Allow read and write on your bucket

For more information about how to perform these steps , check out the Flexible Engine Documentation or your Cloud Provider Documentation.

Step 1: Install s3fs-fuse

If you are using a new Ubuntu instance, you first need to update your system.

Next, install the dependencies.

Then,clone the s3fs source code from git.

Now change to source code directory, and compile and install the code with the following commands:

Use below command to check where s3fs command is placed

Step 2: Configure Credentials

Create a new file /etc/passwd-s3fs and paste your access key and secret key in the below format

Then, change the permission of the file

Step 3: Mount your S3 Bucket as a File System

Now create a directory and mount your Object Storage Bucket in it.

your_bucketname is the name of your Object Storage Bucket that you have created on your Cloud Account.

use_cacheto use a directory for its cache purpose.

allow_otherto allow other users to write to the mount-point.

uid is the uid the owner of the mountpoint(to get your id , enterid on CLI)

mp_umask to remove other users permission.

multireq_maxis a parameter to send request to your Object Storage Bucket.

/mys3bucketis the mountpoint where the bucket will be mounted .

The url for Flexible Engine can be replaced by your Object Storage Endpointprovided in your Object Storage Bucket Basic Information on your Flexible Engine Account.

In order to mount your S3 Bucket automatically after every reboot you should define the script /etc/mount.sh then use crontab -e to automatically run /etc/mount.sh script to remount your S3 after every reboot.

You can run sudo systemctl reboot to reboot your ECS instance.

For debugging purposes , you can add -o dbglevel=info -f -o curldbg in the s3fs mount command.

In order to check if your Object Storage Bucket have been mounted, run the following commands :

You’ll get the following result :

Figure taken from this blog

If it shows the mounted file system, you have successfully mounted your OBject Storage Bucket on your ECS Instance. You can also test it by creating a test file

In order to see the above change, Login to your Object Storage Bucket and verify if the test file is present or not. If it is present then Congrats ! your Bucket is successfully mounted to your ECS instance. From now, any files written to /mys3bucket will be replicated to your bucket.

Notes :

  • If you already have some data in your bucket which is not visible, then you have to set permissions in your ACL through the use of the your Cloud Provider management console for that bucket.

References

How to Mount S3 bucket on EC2 Linux Instance

Mounting an Object Storage Bucket as File System on Oracle Linux

The Startup

Get smarter at building your thing. Join The Startup’s +724K followers.

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store