Shared Storage Part 3 — File Stores
In the last couple of posts, I have talked about block storage and how data is saved to them. You may want to take a quick look if you have not read those. In this post, I wanted to talk a bit about Files System Formats (such as FAT, APFS, EXT, XFS, …), how they work, and how they apply to block storage. In the last post, I talked about indexes used to track which block in a drive holds data for which file. At the core, File Systems are just a standard for how that index is kept and used.
I plan not to delve into any specific file system format but talk about how they operate in general. Which I guess I already did, lol. So let’s go a bit deeper. File Systems, the kind you specify when you format a drive, are local. This means they only exist on and in the context of that drive. This is why when you format a flash drive and load a file to it on one computer, you can access it on the other by simply plugging it in.
This concept can be extended, though. If you have ever looked at a file manager or worked in a terminal window, you would have seen drive letters or even drives that look like folders. This is the operating system-level file system. It’s a virtual construct that sometimes includes everything, including hardware used to make it easy for humans to navigate a computer’s workings. Under the hood, though, each drive is still using its own File System Format.
The concept can also be extended over the network with NAS (Network Attached Storage) using protocols and solutions such as the aptly named Network File System (NFS) and Network Mapped Drives. But this is where things start to get a bit interesting. Network-based systems can be far more than just logical pointers to a drive and its file system format. Becoming an overlay file system, they are able to do things as simple as storing copies of files across multiple servers to as complex as intelligently moving files between higher speed storage and more cost-effective slower or “cold” storage as needed. While this type of abstraction is not exclusively the domain of Network File Systems, it is predominantly only used in the network.
Redundancy for File Systems is a bit more fragmented with the compute in the case of NASs (The actual NAS server software) separated from the storage. For computing, it is usually just a basic HA config with dynamic access to the storage. This is the same as it would be for any other service in something like K8s. For storage, this is handled either at the file level with replication (think rsync) to a different drive array and/or even on another server or at the block level.
Block level protection is much lower than the File System and where things like SnapMirroring come into play, replicating and synchronizing the data stored on a drive in one system with a drive in another one block at a time. Locally we use things like RAID (Redundant Array of Inexpensive Disks). When a file is stored, it is chopped up into equal-sized blocks. Those blocks can then be saved anywhere on the drive independently of any other blocks in a file. If you have not read my post on block storage, it provides a good primer on this topic.
In a RAID configuration, the first block of a file is stored somewhere on one drive, and the next is stored on another. Using two drives, often the slowest things in a computer, in parallel, has a real performance impact. The redundancy comes in when those blocks are also copied to another set of drives, and/or a third drive is used to store a comparison of the two blocks. In this way, we get speed and redundancy. If any drive fails, the other two drives can be used to rebuild the lost data.
The reality is that File System Formats and Network File Systems are really that simple. In my experience, like most things, the devil is in the details, with most of the complexity coming in how these concepts are executed and what features are added along the way.
In the next posts, I hope to talk about Object Stores and how they are both like and different from Network File Systems. As always, if you have any questions or comments, please feel free to email me at any time.
