How I’m (slowly) moving off the cloud with Nextcloud

A walkthrough on how I made my Raspberry Pi a decent self-hosted cloud.

I bought a Raspberry Pi from Amazon with the goal of moving off of Google Photos to a self-hosted photo backup solution that’s nearly as seamless.

My goals for this project:

  • A Nextcloud instance on my internal network which could backup photos from my phone
  • A VPN to access the server from outside my network with Dynamic DNS
  • An internal DNS server to allow me to have an internal subdomain
  • SSL/TLS certificate from LetsEncrypt for the internal domain

There was one thing I had in mind when coming up with these goals: I didn’t just want my personal cloud to be private, I wanted it to be convenient.

My setup

First off, here are the things I needed for my setup:

The most important thing here is a proper power supply. Most phone chargers will work to power the Pi initially, but they’ll fail when they run something that requires additional power.

To start off, I installed Raspbian Stretch Lite on my Raspberry Pi and connected to it via SSH. To avoid duplicating too much of what’s already available online, you can just follow the guide here.

Once I installed the operating system and was able to SSH into it, I ran the following commands to install updates:

sudo apt-get update
sudo apt-get upgrade

And then changed the default password.

passwd

Setting up Dynamic DNS

Most ISPs will periodically change your external IP address. If you want to be able to reliably access your home network through a VPN, you’ll need to set up Dynamic DNS.

Dynamic DNS will allow you to assign a subdomain to your home network and automatically update the assigned IP address if it changes.

Depending on your domain registrar, the following steps will be different, but I use Google Domains so I’ll outline the process for that here.

First, navigate to the DNS section and under ‘Synthetic Records’, select ‘Dynamic DNS’ from the dropdown.

Next, enter a subdomain and click ‘Add’. Make note of the username and password that is generated by Google Domains.

Once you’ve set up your DNS, the next step is to install the ddclient package.

sudo apt-get install ddclient

You can select the default options for each prompt, we’ll be reconfiguring this information in the next step. For username and password, paste in the information you got from Google Domains in the previous step.

Now, open up the ddclient configuration file:

sudo nano /etc/ddclient.conf

Next, replace the content of the file with the following:

protocol=dyndns2
use=web
server=domains.google.com
ssl=yes
login=generated_username
password='generated_password'
test.example.com

Replace generated_username and generated_password with the username and password you copied down earlier. Also, replace test.example.com with the subdomain you set up.

CTRL+X to exit and Y and Enter to save.

Test your configuration and make sure at least one of the lines starting with use=web has your public IP address.

sudo ddclient -query

It should contain a line that looks similar to this:

use=web, web=loopia address is 67.122.333.444

Next, we want to make sure the service is running periodically. Open the following file:

sudo nano /etc/default/ddclient

Change run_daemon="false" to run_daemon="true" if it isn’t set already. CTRL+X to exit and Y and Enter to save.

If everything goes well, start the ddclient service:

sudo systemctl start ddclient

Lastly, we want to add a cron to ensure that the ddclient service is updating. We’ll create a blank file with a cronjob to force ddclient to update:

sudo nano /etc/cron.daily/ddclient

Add this to the empty file:

#!/bin/sh
/usr/sbin/ddclient -force

Make it executable:

sudo chmod +x /etc/cron.daily/ddclient

And that’s it for Dynamic DNS.

Setting a Static IP

For most of the following sections, you’ll need to make sure your Pi maintains the same IP address consistently. For this, you need to set up Static IP or a DHCP reservation.

> Important: If your router has the capability to do a DHCP reservation, do that, it’s much easier. Because every router is different, this is something you’ll have to find online.

If you can’t do a DHCP reservation, you’ll need to set a Static IP on the Pi itself.

To start, open up the dhcpcd.conf file.

sudo nano /etc/dhcpcd.conf

Next, add the following to the end of the file. You’ll need to replace both instances of 192.168.1.1 with your router’s IP address and replace 192.168.1.2 with the IP address you want to set for your Pi.

interface eth0
static ip_address=192.168.1.2/24
static routers=192.168.1.1
static domain_name_servers=192.168.1.1

Also, if you’re using Wi-Fi instead of a wired connection, replace eth0 with wlan0.

Finally, reboot your Pi.

sudo reboot

Setting up Internal DNS

An internal DNS server is what will allow you to set a domain name with a LetsEncrypt SSL/TLS on your NextCloud instance. Technically, you could put an internal IP on your external DNS, but that would reveal information about your internal network and it’s best to avoid that where possible.

To start off, we need to install dnsmasq.

sudo apt-get install dnsmasq

Open up the configuration file.

sudo nano /etc/dnsmasq.conf

You should see a bunch of options that are commented out. Scroll down to the bottom and paste in the following:

no-hosts
addn-hosts=/etc/hosts.dnsmasq
cache-size=150
no-negcache
  • no-hosts tells dnsmasq to ignore your hosts file
  • addn-hosts tells it to look for a different hosts file (we’ll set this up in the next step)
  • cache-size option sets how many names to be cached (you can increase this if you want)
  • no-negcache avoids caching items that return “no such domain” responses from upstream servers

Next, let’s create our hosts.dnsmasq file.

sudo nano /etc/hosts.dnsmasq

In this file, we need to add any hosts that we don’t want being resolved by the upstream DNS (in my case Cloudflare). Your hosts file should look similar to this:

192.168.1.2 test.example.com
192.168.1.2 nextcloud.example.com

Replace all instances of 192.168.1.2 with your Pi’s IP address and replace example.com with your own domain. In addition, replace test.example.com with the Dynamic DNS domain you set up earlier. This will allow you connect to your Pi using the same domain as you use externally.

CTRL+X to exit and Y and Enter to save.

Restart dnsmasq to load the changes.

sudo systemctl restart dnsmasq
sudo systemctl enable dnsmasq

Next, you need to make sure your devices are using the Pi as their DNS server. I’ve included the steps for macOS and iOS below.

For macOS,

  1. Open System Preferences and click on Network
  2. On the left, click the interface you’re connected to (usually Wi-Fi or Ethernet)
  3. Click on Advanced in the bottom-right corner
  4. Select DNS from the top navigation bar
  5. Under DNS Servers on the left, click the + and add your Pi’s IP address to the list of DNS servers your computer should use.

For iOS,

  1. Open the Settings app and select Wi-Fi
  2. Tap on the ⓘ next to the name of your router
  3. Tap Configure DNS and select Manual
  4. Under DNS Servers tap Add Server and enter your Pi’s IP address

For other devices, you should be able to find guides online, but here are sources for Android, Windows and Ubuntu.

Now, if you type nslookup test.example.com from your computer, you should see it return your Pi’s IP address.

$ nslookup test.example.com
Server: 192.168.1.2
Address: 192.168.1.2#53
Name: test.example.com
Address: 192.168.1.2

Setting up OpenVPN

While it was possible to do this manually, I decided to use a simpler setup script created by Stanislas Lange. Tell him thanks by clicking here!

First off, you should cd into your home directory.

cd ~

Next, download the script by running:

curl -O https://raw.githubusercontent.com/Angristan/openvpn-install/master/openvpn-install.sh

Make the script executable.

chmod +x openvpn-install.sh

Now, run the script.

sudo ./openvpn-install.sh

The script will ask for your internal IP address. Most of the time this is pre-filled.

IP address: 192.168.1.2

Next, it will ask for your Public IPv4 address or hostname. This will be the hostname for the dynamic DNS we setup earlier.

Public IPv4 address or hostname: test.example.com

For IPv6 support, leave the default n.

Do you want to enable IPv6 support (NAT)? [y/n]: n

For the next question, I chose a random port instead of leaving the default 1194. It’s not necessary to choose a random one, but it gives you a slightly better chance against port scanners that scan the internet for common network services.

What port do you want OpenVPN to listen to?
1) Default: 1194
2) Custom
3) Random [49152-65535]
Port choice [1-3]: 3
Random Port: 59348

If you didn’t pick the default, note down the port number the script assigned. In my case, it is 59348.

For the next question, leave the default configuration:

What protocol do you want OpenVPN to use?
UDP is faster. Unless it is not available, you shouldn't use TCP.
1) UDP
2) TCP
Protocol [1-2]: 1

Next, you can choose any DNS provider you feel most comfortable with, but I used Cloudflare for their focus on privacy.

What DNS resolvers do you want to use with the VPN?
1) Current system resolvers (from /etc/resolv.conf)
2) Self-hosted DNS Resolver (Unbound)
3) Cloudflare (Anycast: worldwide)
4) Quad9 (Anycast: worldwide)
5) Quad9 uncensored (Anycast: worldwide)
6) FDN (France)
7) DNS.WATCH (Germany)
8) OpenDNS (Anycast: worldwide)
9) Google (Anycast: worldwide)
10) Yandex Basic (Russia)
11) AdGuard DNS (Russia)
DNS [1-10]: 3

I left the next two options as their default:

Do you want to use compression? It is not recommended since the VORACLE attack make use of it.
Enable compression? [y/n]: n
Do you want to customize encryption settings?
Customize encryption settings? [y/n]: n

After the script installs the required packages and dependencies, it will ask you for a client name. This is the name of the first user you want to add to your VPN server.

Client name: vpnuser

The script will also ask you if you want to add a password to the client. This allows you to encrypt the private key generated with a password.

This isn’t required, but there’s no harm in encrypting your key since most clients will let you save the password and you won’t have to enter it repeatedly.

Do you want to protect the configuration file with a password?
(e.g. encrypt the private key with a password)
1) Add a passwordless client
2) Use a password for the client
Select an option [1-2]: 2
Enter PEM pass phrase:
Verifying - Enter PEM pass phrase:

Next you need to install the OpenVPN app for iOS or Android on your device and copy over the certificate. Once you have the app downloaded, enter the following command on your Pi to start a simple web server you can access from your phone:

python -m "SimpleHTTPServer"

Now, on your phone, open your web browser and navigate to http://192.168.1.2:8000 (replace 192.168.1.2 with the IP address of your Pi) and download the .ovpn file and add the profile to your OpenVPN app.

CTRL+C out of the Python web server once you’ve transferred the file.

Open the OpenVPN configuration file.

sudo nano /etc/openvpn/server.conf

Add the following above the line starting with push "dhcp-option(Replace the IP address with your Pi’s IP address).

push "dhcp-option DNS 192.168.1.2"

Restart OpenVPN.

sudo systemctl restart openvpn

The last step is to set up port forwarding on your router and forward the OpenVPN port you noted down earlier to the local IP address of your Pi. Because everyone has a different router, I’ll just link a generic guide here.

Once port forwarding is set up, you should be able to connect to your VPN.

Setting up the web server

The first step is to install Apache and enable it to start on boot:

sudo apt-get install apache2
sudo systemctl enable apache2

Next, we need to install MySQL (MariaDB) and enable it to start at boot:

sudo apt-get install mariadb-server
sudo systemctl enable mariadb

To help secure your MySQL instance, run the following and answer Y to the prompts. This will help you set a root password and remove unnecessary tables from your server.

sudo mysql_secure_installation

Your terminal should look a little bit like this:

Enter current password for root (enter for none):
Set root password? [Y/n] Y
New password:
Re-enter new password:
Remove anonymous users? [Y/n] Y
Disallow root login remotely? [Y/n] Y
Remove test database and access to it? [Y/n] Y
Reload privilege tables now? [Y/n] Y

Next, you need to install PHP:

sudo apt-get install php7.0 libapache2-mod-php7.0

You’ll also need some additional PHP packages:

sudo apt-get install php7.0-common php7.0-gd php7.0-json php7.0-mysql php7.0-curl php7.0-mbstring php7.0-intl php-imagick php7.0-xml php7.0-zip php7.0-mysql

Setting up LetsEncrypt

LetsEncrypt is a free certificate authority that’ll let you have SSL/TLS certificate for your Nextcloud instance.

First, move into your home directory and download the latest version of Certbot, an automated certificate setup tool, to your Pi.

cd ~
sudo wget https://dl.eff.org/certbot-auto

Make the script executable.

sudo chmod a+x certbot-auto

Next, run the following (replace example.com with your own domain).

sudo ./certbot-auto certonly --manual --preferred-challenges dns-01 -i apache -d "*.example.com" --server https://acme-v02.api.letsencrypt.org/directory

You may be asked to install some additional dependencies. Enter Y if prompted. (This may take a while).

First, enter the email address you’d like to receive important information at:

Enter email address (used for urgent renewal and security notices) (Enter 'c' to cancel): email@example.com

You’ll also need to read and agree to the terms of service:

Please read the Terms of Service at
https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf. You must agree in order to register with the ACME server at
https://acme-v02.api.letsencrypt.org/directory
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
(A)gree/(C)ancel: A

The next one is up to you.

Would you be willing to share your email address with the Electronic Frontier Foundation, a founding partner of the Let's Encrypt project and the non-profit organization that develops Certbot? We'd like to send you email about our work encrypting the web, EFF news, campaigns, and ways to support digital freedom.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
(Y)es/(N)o: N

To obtain a certificate, you must agree to the IP logging.

NOTE: The IP of this machine will be publicly logged as having requested this certificate. If you're running certbot in manual mode on a machine that is not your server, please ensure you're okay with that.
Are you OK with your IP being logged?
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
(Y)es/(N)o: Y

Next, you need to add a TXT record to your external DNS. This allows LetsEncrypt to verify that you actually own the domain you’re trying to obtain a certificate for.

Please deploy a DNS TXT record under the name
_acme-challenge.example.com with the following value:
KYL9wfbBTif3bfdMZ7xwwqxWgXsvLVNPePmuxGEmnMY
Before continuing, verify the record is deployed.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Press Enter to Continue

> Important: Before pressing Enter, take the value displayed by certbot and add it to your external DNS. I’ll show how to do that for Google Domains, but the process should be relatively similar for most registrars.

Open the DNS section for Google Domains.

Scroll down to ‘Custom resource records’ and and select TXT from the dropdown. Next add _acme-challenge to the name field on the left and in the data field, paste the value from the certbot script.

Click ‘Add’ once you’re done.

> Important: Wait for a few minutes for the DNS to propagate. If you don’t wait long enough, the script will fail and you’ll have to start this process from the beginning.

Now that you’ve waited, you can press Enter and continue the script.

Waiting for verification...
Cleaning up challenges
IMPORTANT NOTES:
- Congratulations! Your certificate and chain have been saved at:
/etc/letsencrypt/live/example.com/fullchain.pem
Your key file has been saved at:
/etc/letsencrypt/live/example.com/privkey.pem
Your cert will expire on 2019-04-30. To obtain a new or tweaked
version of this certificate in the future, simply run certbot-
auto again. To non-interactively renew *all* of your
certificates, run
"certbot-auto renew"
- If you like Certbot, please consider supporting our work by:
Donating to ISRG / Let's Encrypt:   https://letsencrypt.org/donate
Donating to EFF: https://eff.org/donate-le

Now that we’ve downloaded the certificates, it’s time to link them to Nextcloud.

Installing Nextcloud

First, we need to create a database user for the Nextcloud instance.

Enter your MySQL server and enter the root password you created in the previous section:

sudo mariadb -u root -p

Run the following commands to create a new database user. Be sure to change ‘password’ to something secure!

MariaDB [(none)]> CREATE DATABASE nextcloud;
MariaDB [(none)]> GRANT ALL PRIVILEGES ON nextcloud.* TO 'nextcloud'@'localhost' IDENTIFIED BY 'your-mysql-password';
MariaDB [(none)]> FLUSH PRIVILEGES;
MariaDB [(none)]> exit;

Now it’s time to download Nextcloud.

Head over to https://download.nextcloud.com/server/releases/ and copy the url for the latest version of Nextcloud in .tar.bz2 format. For me, this was nextcloud-15.02.tar.bz2.

To do this, just navigate to the link above, right click the file and click ‘Copy Link Address’.

Next, you’ll want to download and extract that to your Pi’s web directory. Switch to the web directory by entering:

cd /var/www/

and then run the following to download and extract Nextcloud.

sudo curl https://download.nextcloud.com/server/releases/nextcloud-15.0.2.tar.bz2 | sudo tar -xj

Now, we need to change ownership of the nextcloud folder to the web user and move into that directory:

sudo chown -R www-data:www-data nextcloud
cd nextcloud

Run the following command to set up Nextcloud with MySQL using the password you set before. Again, be sure to change the Nextcloud administrator’s password to something more secure.

sudo -u www-data php occ maintenance:install --database "mysql" --database-name "nextcloud" --database-user "nextcloud" --database-pass "your-mysql-password" --admin-user "admin" --admin-pass "nextcloud-admin-password"

You should see this:

Nextcloud was successfully installed

We should run the following command to remove the plaintext password we just entered from our bash history:

history -d $(history 2 | head -n 1 | awk '{print $1}')

Next, we need to set up Apache Virtual Hosts to access our Nextcloud instance.

Start by opening an empty configuration file:

sudo nano /etc/apache2/sites-available/nextcloud.conf

Paste the following into the file, but replace all instances of example.com (in bold) with your own domain.

<VirtualHost *:80>
ServerAdmin email@example.com
ServerName nextcloud.example.com
ServerAlias www.nextcloud.example.com
DocumentRoot /var/www/nextcloud
ErrorLog ${APACHE_LOG_DIR}/error.log
CustomLog ${APACHE_LOG_DIR}/access.log combined
    RewriteEngine On
RewriteCond %{HTTPS} off
RewriteRule ^/?(.*) https://%{SERVER_NAME}/$1 [R,L]
</VirtualHost>
<IfModule mod_ssl.c>
<VirtualHost *:443>
ServerAdmin email@example.com
ServerName nextcloud.example.com
ServerAlias www.nextcloud.example.com
DocumentRoot /var/www/nextcloud
ErrorLog ${APACHE_LOG_DIR}/error.log
CustomLog ${APACHE_LOG_DIR}/access.log combined
SSLEngine on
SSLCertificateFile /etc/letsencrypt/live/example.com/cert.pem
SSLCertificateKeyFile /etc/letsencrypt/live/example.com/privkey.pem
SSLCertificateChainFile /etc/letsencrypt/live/example.com/chain.pem
</VirtualHost>
</IfModule>
SSLProtocol all -SSLv2 -SSLv3
SSLCipherSuite ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECD$
SSLHonorCipherOrder on
SSLCompression off

There’s a lot in here, but essentially, this configuration file tells Apache to:

  • look for the site in the /var/www/nextcloud folder
  • redirect http traffic to https
  • use the SSL/TLS certificates located in /etc/letsencrypt/live/example.com
  • disable weak SSL ciphers such as SSLv2 and SSLv3.

CTRL+X to exit and Y and Enter to save.

Next, we need to enable a couple of apache modules.

sudo a2enmod ssl
sudo a2enmod rewrite

And finally enable the site and restart Apache.

sudo a2ensite nextcloud.conf
sudo systemctl restart apache2

You should now be able to access your Nextcloud instance at nextcloud.example.com from any computer on your network that’s connected to your Pi DNS.

You can now log in with the administrator account and create a separate user account for yourself and change any settings you want to change.

Be sure to download the iOS or Android app and set up photo backup!

Setting up External Storage (Optional)

Now, everything on Nextcloud should be working, but adding external storage makes Nextcloud more usable. You can store much larger files without having to worry about remaining space on your SD card.

> Important: This will erase all data on your drive. If you have any important data, please back it up before starting this process.

First, you have to connect and format your drive. Plug the drive into a powered USB hub and connect the hub to the Pi. List all the connected disks:

sudo fdisk -l

You should see something like this:

Disk /dev/sdx: 1.8 TiB, 2000365289472 bytes, 3906963456 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

Now, note down the disk name (mine is /dev/sdx) and replace that in the next command. It’s important that you get the correct value here because you can accidentally wipe the wrong disk if you don’t.

Run the following to wipe and format your disk (replace /dev/sdx with your drive name):

sudo fdisk /dev/sdx 

You should see the following:

Welcome to fdisk (util-linux 2.29.2).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Enter d in the first prompt.

Command (m for help): d
Selected partition 1
Partition 1 has been deleted.

Next, enter the following (in bold) to add a new partition (note: leave the first sector and last sector prompts blank):

Command (m for help): n
Partition type
p primary (0 primary, 0 extended, 4 free)
e extended (container for logical partitions)
Select (default p): p
Partition number (1-4, default 1): 1
First sector (2048-3906963455, default 2048):
Last sector, +sectors or +size{K,M,G,T,P} (2048-3906963455, default 3906963455):
Created a new partition 1 of type 'Linux' and of size 1.8 TiB.

Enter w to write the changes to your disk.

Command (m for help): w
The partition table has been altered.

Now, we need to format the disk as ext4. Run the following and be sure to replace /dev/sdx1 with the drive name we used in the previous commands.

sudo mkfs.ext4 /dev/sdx1

Next, we need to set a location to mount the drive to. Create folder called external in the media directory.

sudo mkdir /media/external

Enter the following to mount the drive (again replacing /dev/sdx with your drive name) and cd into that directory and create a nextcloud folder.

sudo mount /dev/sdx1 /media/external/
cd /media/external
sudo mkdir /media/external/nextcloud

Let’s move our Nextcloud data folder into this drive. We need to move the folder and create a symbolic between the original location and the new one.

sudo mv /var/www/nextcloud/data/ /media/external/nextcloud/
sudo ln -s /media/external/nextcloud/data /var/www/nextcloud/data

Next, we want to ensure that the web server can access the data directory.

sudo chown www-data:www-data /media/external/nextcloud/data

Load up nextcloud.example.com to make sure it’s still working.

Finally, we need to make sure the drive is mounted on each boot. Enter the following to list out your connected drives.

sudo blkid

Copy down the UUID for your external drive.

/dev/sdx1: UUID="e4957cf5-0290-aa6c-41b4-b8673f8da3a9" TYPE="ext4"

Open up your fstab file.

sudo nano /etc/fstab

Paste the following at the bottom of the file. Be sure to replace the UUID of my drive with yours.

UUID=e4957cf5-0290-aa6c-41b4-b8673f8da3a9 /media/external ext4 defaults,nofail 0 0

The UUID makes sure that if you connect your devices in a different order, it will still mount to the same directory.

Reboot your Pi to finish up.

sudo reboot

Wrapping Up

We’ve added a lot of different services to our Pi, but now, you should be able to connect to your Nextcloud instance from your home or over VPN and backup your photos and other files.

For maintenance, the only things you need to do are renew your LetsEncrypt certificates (every 90 days) and update Nextcloud and your Pi to the latest versions to keep up with security patches.