How to Build a Personal File Sharing and File Syncing System With Raspberry Pi

The Reason Why:

A number of years ago, I went looking for a NAS system that was good and affordable. I found a good drive at a decent price, but what I didn’t know until I was already invested was that the software that ran it was highly flawed and limiting. As time has gone on, different cloud options have proliferated, most of them involving some exchange of money or data security, or both, for convenience.

That brings us to my project: creating my own “cloud”, a computer on my home network that hosts all my media files and also syncs my important files on any number of devices on my network. This can all be accomplished with fairly low-cost components and free software.

The main purpose of this article is to offer a distilled and concise step-by-step how-to, start to finish, for this project.

The Ingredients:

• A Raspberry Pi of your choosing (I would recommend at least a 3 or a 4 for this project, although if you’re not serving/syncing much, you might be able to get away with less); for this project, I used a Raspberry Pi 3 B+.
• The appropriate power supply, either for an RPi Zero-3 or an RPi 4. Don’t skimp on this; RPis don’t use a lot of power, but they need it steady, and you shouldn’t try to just run it on any old Android phone power supply.
• A MicroSD card with at least 4GB; there are methods for booting a Raspberry Pi off of a 2GB card that are beyond the scope of this article. I recommend using 8GB at least, just to be safe. In my project I ended up with a 32GB Samsung MicroSDXC EVO.
• To be able to interact with the Raspberry Pi: A screen with HDMI input, a mini-HDMI to HDMI cable, USB keyboard and mouse OR the ability to SSH into the Raspberry Pi
• A computer with an SD card slot and a MicroSD to SD adapter OR a computer with a MicroSD slot
• A hard drive of your choosing. I highly recommend an SSD drive, rather than an HDD drive, because of speed and reliability, both of which are very important in this application (although you could use a simple USB stick, nowadays there’s some with really big capacity). Because I was using parts that I had around for this project, I used a 3TB Western Digital “red” NAS drive, which works just fine. Keep in mind that Raspberry Pis don’t have a lot of extra electricity to power external drives, so you may need an external power source. I used a Vantec IDE/SATA to USB 3.0 Adapter and it worked a treat.
• This article assumes you are interacting with Windows computers that are set up in a network (workgroup); my wife’s Mac laptop is able to interact with that workgroup, as is my Android phone. However, if you have a solely Mac or Linux ecosystem, you will need to do additional research.

And, of course:

• A steady internet connection with a wireless router (or a lot of ethernet cables)

The How

Part 1: Get your Raspberry Pi up and running

This is covered in great detail by people who know a lot more than me elsewhere, and some searching will take you far. If you get lost, here’s the reference I use. But my workflow is as follows:

• On my computer with an SD or MicroSD slot, install the Raspberry Pi Imager.
• Insert the SD card and install the 32-bit Raspberry Pi (Raspbian) OS:

Of course, you can choose a different OS if you’re doing something more custom (headless Ubuntu, for example)

Once it’s done writing and checking, insert the SD card into the RPi, make sure all your peripherals (including the hard drive you’re planning on networking!) are plugged in, powered up and ready to go, and boot up that Pi!

Part 2. Getting the hard drive mounted and properly permissioned

First of all, open up a terminal window and type the following bash command to find out where the hard drive is mounted:

df -h

You should get output that looks like the following. Take a look at the line, almost at the bottom, that begins ‘/dev/sda4…’:

pi@raspberrypi:~ $ df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/root        14G  3.4G   11G  25% /
devtmpfs        459M     0  459M   0% /dev
tmpfs           464M     0  464M   0% /dev/shm
tmpfs           464M  7.3M  456M   2% /run
tmpfs           5.0M  4.0K  5.0M   1% /run/lock
tmpfs           464M     0  464M   0% /sys/fs/cgroup
/dev/mmcblk0p1  253M   51M  202M  21% /boot
/dev/sda4       2.7T  1.4T  1.3T  53% /media/pi/3.0 TB Volume
tmpfs            93M     0   93M   0% /run/user/1000
pi@raspberrypi:~ $

That’s the drive I’m interested in; make a note of it. What I’m going to do now is to assign to it a permanent, custom, mount point. We’ll get more into the technical reason why later, but the idea is I create a directory, let’s call it /network, in my root directory. That entire directory is going to get served to the network. Therefore, within that directory, I’ll make a second directory where I will mount my hard drive; that way, I can always add another drive later on, also mounted inside /network, without having to change the Samba specifications. First of all, let’s create these directories. Step one, get to the root of your filesystem (of course, you can create this directory wherever you want):

cd /

And create the directory to be served:

sudo mkdir network

cd into it:

cd network

And create the directory for the drive itself:

sudo mkdir NetworkDrive

There can be big problems in terms of actually using the network drive if it doesn’t have the right permissions. This is a big, complex issue and if you have an NTFS drive, this will be an area where you will need to do further research (see below).

For my purposes (ext4 filesystem, likely works on other Linux filesystems), I imposed wide-open file permissions so that any device on the network could read, use, edit, delete, etc. any file (replace ‘/network/NetworkDrive’ here with your own directories):

sudo chmod -R 1777 /network/NetworkDrive

OK. In theory, you should have a custom set of directories with wide-open permissions, ready to have the drive mounted to them and then put on the network.

Next step: set specifications so that the drive automatically loads to your specified directory. In order to do that, I need to know some specifics about the drive. So, let’s ask the system for that information:

sudo blkid /dev/sda4

Do the same on your system, substituting ‘/dev/sda4’ for the path to the hard drive you want to mount. You should get something like:

/dev/sda4: UUID="6b7befea-32df-495d-b3d6-e315960b04e2" TYPE="ext4" PARTLABEL="primary" PARTUUID="B4b63cac-91f1-42e8-0068da23091e"

We care about two things, here: the UUID, which is the specific ID of the hard drive, and the TYPE, which is the filesystem encoding on the drive. Make note of both these things. There are a number of different filesystem types; mine, here, is ext4, which is a Linux type; if you have NTFS, which is Windows, you will have to do some extra work but the good news is it’s a very normal thing. This article has some great tips towards the bottom about dealing with NTFS filesystems; if that link has gone dead, try searching for “how to mount NTFS hard drives on Raspberry Pi”.

Now, armed with our UUID and filesystem type, we can finally set the custom mount point for the drive. To do that, we’re going to open a file called fstab:

sudo nano /etc/fstab

It should have a few entries already, such as:

proc              /proc          proc    defaults           0      0
PARTUUID=a18a5503-01  /boot          vfat     defaults         0  2
PARTUUID=a18a5503-02  /              ext4     defaults,noatime 0  1
# a swapfile is not a swap partition, no line here
#   use dphys-swapfile swap[on|off]  for that

What we’re going to do is set up a rule so that every time the RPi boots, it will mount the drive at a location that we specify.

proc              /proc          proc    defaults           0      0
PARTUUID=a18a5503-01  /boot          vfat     defaults         0  2
PARTUUID=a18a5503-02  /              ext4     defaults,noatime 0  1
UUID="6b7befea-32df-495d-b3d6-e315960b04e2" /network/share ext4 auto,rw,user,nofail 0 0
# a swapfile is not a swap partition, no line here
#   use dphys-swapfile swap[on|off]  for that

fstab can be confusing — lots of columns that don’t seem to add up, specifications without documentation — but the important thing is that you put all the elements in: the UUID, the mount point you’re specifying, the filesystem type, the additional specifications I’ve listed above (that’s important), with at least one space between each item.

OK; hit ‘ctrl+X’ to exit Nano, save your changes, and get ready to make this thing go live. The next command will run all the drive specifications in fstab:

sudo mount -a

Ok, great. Navigate to your custom directory, and you should see the contents of your drive there; if you’r using a fresh drive, try creating a file or another folder inside it; you should be able to. If you can’t, it’s time to start troubleshooting permissions.

Part 3. Get your drive serving to your network

This is the home stretch! Firstly, you will need to install Samba, which is an incredibly powerful and flexible file serving program with a huge development community surrounding it:

sudo apt-get install samba

Once that’s done, we are going to open the configuration file for Samba:

sudo nano /etc/samba/smb.conf

All the way at the bottom, you will need to create a definition for the ‘share’ you’re creating. Again, there is an incredible wealth of options out there, and the official Samba documentation has exhaustive defintions. If you’re following Samba tutorials off the internet, be sure to double-check details: the first tutorial I followed spelled ‘browseable’ without the first ‘e’, which causes problems. This is the set of specifications I used:

[NetworkDrive]
path = /network
browseable = yes
writeable = yes
public = yes
guest ok = yes
read only = no
create mask = 0777
directory mask = 0777

Boom. That’s it. Hit ctrl+x, save your changes, and restart Samba. This command is different on different flavors of Linux, and on RPi the command that works for me is:

sudo systemctl restart smbd

OK. Give it a few seconds if you want, but you should have a Samba share live on your network. So, there you go: you now have a hard drive mounted on your network, and all the devices on that network should have full permissions on those files. Stop here if you want — but if you want to auto-sync your files across devices… Follow me further down the rabbit hole:

Part 4. Install Syncthing

There are other filesyncing options out there, including Resilio Sync. Personally, I appreciate the open-source, cross-platform ethos of Syncthing. The rest of the procedure is as follows:

curl -s https://syncthing.net/release-key.txt | sudo apt-key add
sudo apt-get update
sudo apt-get install syncthing

Syncthing should install following this. If it doesn’t, then likely something has changed with the URL, and you should research a current way of installing syncthing on RPi.

So there’s nothing more to it: run syncthing!

syncthing

On RPi, syncthing runs in the background, and you can interact with it by opening a browser and navigating to 127.0.0.1:8384.

Here, you can choose which folders to sync between your devices; I have a folder containing many of my essential documents, and on my RPi it syncs to the same network-mounted drive we set up above. Just like Samba, syncthing is a sprawling project with lots of ins and outs, so take a jaunt through the documentation to familiarize yourself with how it works.

An important caveat: this tutorial assumes you want everyone on your network to have access to everything. I’ve referred to this previously, but it bears stating specifically that there are numerous ways to restrict access to different parts of your file sharing and serving system. You can implement these restrictions in Raspbian, at the Linux file permissions level; you can implement them in Samba (take another look at those share definitions) and you can even implement restrictions in syncthing. Consider carefully your method, and the scope of it.

That’s it! If you made it this far, enjoy your new media center/file sharing/backup system, built by your own hand and made to your specifications.

Stay tuned, though: the next project? A Pi Zero-powered remote node of the synced files. That’s right… A portable private cloud.