Building a Fuse File System
Alright, admit it. The reason why you’re reading this is because you have no idea how FUSE works. You have read the documentation over at the Libfuse repository and even looked at some example file systems but it still doesn’t click. I too was in a similar situation as yourself.
During my final year of studying Computer Science at the University of California Riverside, I decided to take Operating System’s as my Senior Design project. I had no choice tbh, no other course was offered and in all honesty I hated Operating Systems. My professor pitched several projects students could select. Looking to avoid any low level projects, I went all in with creating a FUSE file system. Oh boy, did I regret taking that class. It was a semester full of pain, anguish, and fear. After all, it was my final semester at UCR. Through the pain of it all, I actually Ace’d the course. My partner and I were able to build a Peer to Peer File System thanks to FUSE and OpenDHT.
Now that you know my experience regarding FUSE am sure you’re wondering how FUSE works. Well I’ll give you the abridged version of working with FUSE. I suffered too much to just give you everything so I’ll just give you a high level run down with resources that will be of great help.
All explanations will be non technical with links to the full details.
Installing FUSE Libraries
For those who don’t want to read a full story of useful tips and just want to get up and running below is the installation process of FUSE.
If you just want to build the FUSE library you can follow the FUSE installation instruction on my P2P FileSystem Readme. The installation process is pretty much identical to that on LibFUSE repo so go there for the official installation if a new version is up. Make sure to only follow the FUSE installation, anything else would go out of the scope of FUSE specifically. I recommend this route instead of just using a package manager because the tests at the end display whether or not you’re system can run FUSE. You don’t need all of the tests to pass, just a majority.
The official FUSE Github Repo
https://github.com/libfuse/libfuse
Back to learning a bit about FUSE.
What is FUSE?
FUSE is a means to create a fully functioning file system in a Linux distribution that has the FUSE kernel module. That last part is important. When I first started researching about FUSE I realized this would be a Linux only project.
I had experience working with Virtual Machines but after using the Linux Subsystem for Windows, I had grown spoiled. Unfortunately for me, I had spent far too long trying to build FUSE from the libfuse repo only to realize the Linux Subsystem for Windows did not have the FUSE kernel module. I would have to get Virtual Box and install Ubuntu to continue working on the project.
The FUSE kernel module is pretty important. Regularly when you click on folders, enter sub directories, rename files, and more System Calls are made. FUSE intercepts these System Calls and allows you to insert any functionality you want through specifically named functions. Whenever you open a folder FUSE intercepts the System Call that opens the folder and instead calls a FUSE function called something like OpenDir(). You put code there specifying what should happen during the call and then return something. Sometimes what you return is valuable data, other times just a flag signifying the operation went well or not. Something fundamental to keep in mind, FUSE intercepts System Calls and does not run them, if you want OpenDir() to actually open a directory you will have to use the System Call that does that operation or use some other System Call that does some another operation.
You can see how this can lead to a diverse amount of File Systems. Some people can make a file system that closes a directory when you try to open one, others can do something entirely different. Some people have built FUSE file systems that are dedicated to music files and as you have seen with my FUSE file system, one that doesn’t even host files locally. What you want you’re file system to do will depend on what you program each function to do when a system call is made.
For an indepth list of all the FUSE functions, check out this IBM list that displays them all along with descriptions of what they do.
While that IBM link offers valuable information it still doesn’t list all the FUSE functions. Just some of the essentials. There are three more resources that will give you a better understanding of what’s going on. The first is the simple yet stupid file system.
Simple Yet Stupid
This tutorial will give you the absolute basics of FUSE without needing to do it the official way. The take away here is that you learn that whenever a system call is intercepted, you decide what the File System should do. In the case of the SYSFS, the author decides to just replicate regular file system operations. You’ll see instead of storing files in memory or using file handler’s, the author instead uses standard data structures. Pretty neat dive into FUSE that really gave me a firm understanding of what was happening.
Man oh Man
The next resource is probably one that will make your eyes roll but shouldn’t be overlooked. Man Pages. Yes Man Pages, the simple little descriptive pages that display information of code snippets. The Stack Overflow before Stack Overflow? Anyways Man Pages were really helpful when trying to understand what was going on. As stated before, I am not an OS fan. Anything low level is something I didn’t want to dive to deep into. What saved me was the Man Pages available online. FUSE functions pass in variables and use System calls I just wasn’t familiar with. Whenever I cam across some System Call I didn’t know I would google the name of the function, find it’s man page and read it’s description. Want to know how to open a directory with a System Call? Bam there’s a Man Page explaining the required syntax and what you need to pass in to the call itself.
Big Brother is Watching
The most helpful of all FUSE resources is the Big Brother File System. I can’t tell you enough how important this FUSE file system was to understanding everything. As it’s namesake implies, the Big Brother FS logs any operation committed within the File System.
The author of the file system Joseph J. Pfeiffer, offers a fully functional file system that is buildable on your Linux machine. Unlike most other FUSE file systems in the libfuse Repo, this one is straight forward and to the point. Each function does it’s respective function during a system call and then logs the operation. Go to his site, read and go through his tutorials which should get you up and running but most importantly, go through each function, when you don’t understand what is being passed in to the function or what a system call is doing go to its Man Page and read up how it works and write it down maybe in a comment above the function. This will give you insights of the background processes going on like File Descriptors and how the buffers work when reading or writing to a text document.
I couldn’t have created my own P2P File System had I not emailed Joseph for some consultation of my own.
Well I hope these resources offered you several placed to sink your teeth into your projects. It will take some work and time but eventually you’ll have something unique to show off.
If you found any of this useful please consider supporting me through ko-fi.
