Get SSH Working on a Raspberry Pi 4

Eric North
Dec 1, 2020 · 15 min read

Secure shell (SSH) is a very handy tool for connecting to your Raspberry Pi, especially if you do not have a monitor, keyboard and mouse at your disposal. Think of SSH as a remote log-in which allows you to work on another system potentially far-removed from the comfort of your easy chair, office chair, standing desk, granite slab or whatever other work area you chose.

Wait, hold on: I might have heard about SSH but what is this ‘Raspberry Pi’, anyway? Dear readers, allow me to present:

Raspberry Pi single-board computer.
Raspberry Pi single-board computer.
Courtesy of Quentin Schulz on UnSplash

Think of the Raspberry Pi as a tiny computer running Raspbian, a flavour of Linux. A tasty flavour, I might add! Because of the Pis’ Linux roots, many of the steps in this article therefore apply to configuring SSH on a much bigger host; Debian boxes to be specific. The instructions should work well with other types of Linux operating systems but your mileage may vary.

There are a few tricky things I encountered when I first attempted to set up secure shell on my RPi. SSH is the gift that keeps on giving! I ran into several difficulties during my intial setup and I suspect other users will encounter similar issues. There are a decent number of rather straightforward tutorials that explain how to SSH into a RPi including this one and that one. Along with these tutorials I found that I had to dig around a couple of discussion posts when I ran into difficulties. I hope that this article will help others in their quest to connect to a Pi via SSH!

Server Setup

You will also want to change any items marked with <..> to suit your particular situation.

Make sure your SuperUser has a strong password before proceeding any further! The default password when logging into a Raspberry Pi for the first time needs to be changed to something stronger before you make additional changes.

Assuming you have all of the above pieces, be sure to perform an initial set-up of SSH once you are logged into Raspbian. You will want to verify SSH enabled first and foremost. Navigate to the ‘raspberry’ icon in the menu (normally at the top-left) and select Preferences, Raspberry Pi Configuration. Enter your SuperUser password when prompted. In the Interfaces tab, select Enable for SSH. Close the menu.

If for some strange reason you do not have SSH installed on your Raspberry Pi you can use the following steps to install it; see this post for details. Within a terminal on your Pi type:

sudo apt install openssh-server

Confirm SSH is running using the following command:

sudo service ssh status

Client Setup

ssh -v -o PreferredAuthentications=password -o PubkeyAuthentication=no pi@<pi_IP>

Provided this is the first time on this machine that you have attempted to SSH to your Pi, you should get a prompt asking you whether or not you trust the host, you can say yes to this. Further details are available at this Raspberry Pi forum.

Upon entering your password, you will likely get the dreaded:

Permission denied please try again while logging in with SSH.
Permission denied please try again while logging in with SSH.
Dum dum dummmmm!

Well then.

Here is the part where I get a little fist-shakey at my Pi. But do not dispair, there are some things we can do to get SSH working.

Setup — Digging Further

With periphirals connected and your host (Pi) restarted, go ahead and log-in. Pull up a terminal session. You can now inspect your host’s logs for any error messages. SSH is rather quiet when it comes to informing users of errors. Any such errors printed to the terminal may appear vague; this is in all likelyhood to prevent malicious users from getting the upper hand and gaining access to your system. No one wants that. Especially me!

Inspect your SSH log file using the following command:

sudo nano /var/log/auth.log

Once you have examined your log file for errors related to SSH, you can also inspect the settings file for the SSH daemon. From the terminal on your host type:

sudo nano /etc/ssh/sshd_config

Be sure you are editing sshd_config, NOT ssh_config; see this post.

Enable PublicKey

# Modified YYYY-MM-DD by <author>.
PubkeyAuthentication yes

Uncomment the PasswordAuthentication line as well:

# Modified YYYY-MM-DD by <author>.
PasswordAuthentication yes

When making changes to your configuration files I recommend inserting a comment directly above the change, along the lines of:

# Modified YYYY-MM-DD by <author>.

AllowUsers and AllowGroups

AllowGroups root ssh

For the purpose of this tutorial I recommend that you comment out the AllowGroups line to limit the configuration for the SSH daemon to a single user.

Add an AllowUsers line directly below PasswordAuthentication as follows:

# Modified YYYY-MM-DD by <author>.
PasswordAuthentication yes

# Added YYYY-MM-DD by <author>.
AllowUsers pi

Comment-out AllowGroups if it is present:

# Changed on YYYY-MM-DD by <author>.
# AllowGroups …

Feel free to change the username to whatever name you have set up for your user although be sure that this user is given privileges for SSH. For an initial setup, I find it easier to use the default user pi for troubleshooting. Further details on AllowUsers and AllowGroups as well as their precedence can be found at this helpful link.

You can further specify the host that your user is allowed to connect with. Instead of specifying the user, try specifying <user>@<remotehost> in your list of AllowUsers as suggested by this article:

AllowUsers pi@raspberrypi

Disable PermitEmptyPasswords

#PermitEmptyPasswords no

Becomes:

PermitEmptyPasswords no

Once you are satisfied with the changes, go ahead and save the file by hitting cntl-o then cntl-x to exit.

All good, right? Can we start SSH-ing yet?

I think it is prounounced ‘shush-ing’. And the answer is No.

Setup — Almost There

sudo systemctl restart ssh.service

I am not going to lie, this step also tripped me up a few times!

Now we should be ready to attempt our first connection via SSH. From the terminal on your Pi type:

ssh -v pi@<pi_IP>

Some of you may be asking: ‘Now wait just a darned minute, why would I want to SSH from within my Pi?!?’ Well, if you are like me there is a good chance that your host and client will not be located in the same room. I must be a sucker for punishment. If this describes your situation, it is therefore much more convenient to test whether or not SSH is working by logging in from the host for starters.

If all goes well you should see the following good news in your terminal:

Authentication succeeded while logging in with SSH.
Authentication succeeded while logging in with SSH.
Success!

Provided you get the authentication succeeded message you should be able to log-in to your Pi via SSH from your client machine. Go ahead, give it a try!

When logging in from your client you may still get errors. Inspect your SSH log file using the following command:

sudo nano /var/log/auth.log

Your our log may contain an error such as:

Invalid user <username> from server_IP port <port>

If this is the case, verify you are specifying the correct username in your config file for AllowUsers and try logging in again.

Troubleshooting

  1. Ensure your keyboard layout is correct between both the Raspberry Pi and your client machine (do not use a UK layout on your client when using an American-layout on the host, for example). On my Ubuntu machine I can check my keyboard layout at the top right-hand corner. At the risk of sounding foolish, this has caused me more than a tiny bit of frustration the odd time when I go to log-in to something with a password only to discover my keyboard layout is speaking a different language!
  2. From your client machine, run: dpkg-reconfigure openssh-server; see here for further details on this configuration step.
  3. Verify that you have provided your Raspberry Pi with a static IP address in your router.
  4. Ensure you are logging into SSH with the correct username; you can verify the username once you are logged into your Pi. See the point I made just a few moments ago about troubleshooting using pi as the default user.
Option to change language settings in Ubuntu.
Option to change language settings in Ubuntu.
Look at all those puffy clouds!

Hardening Your Pi with a PublicKey

Important: These next steps will remove and re-install SSH keys on both your client and server. Please please please: check with your System Administrator first, particularly if your server and/or client’s SSH keys are configured for several other machines!

Delete Older SSH Keys

Navigate to the hidden folder on your Pi containing the SSH keys:

cd .ssh

Next, edit the authorized_keys file:

sudo nano authorized_keys

Remove the entries from this file, they will likely be rather long so be patient when removing them.

Hit cntl-o then cntl-x to exit. Delete the authorized_keys file rather than messing with individual keys, provided that your System Admin says it is okay to do so. Bring them a nice coffee or tea if this is your second or third visit!

Next, go to your client and remove the keys, see this post to remove the hidden SSH folder on the client:

rm -rf /home/<client_user>/.ssh

Your SSH keys should now both be cleared from the server and client.

Generate New SSH Key on the Client

ssh-keygen

You may get the following message:

Generating public/private rsa key pair.
Enter file in which to save the key (/home/<client_user>/.ssh/id_rsa):
/home/<client_user>/.ssh/id_rsa already exists.
Overwrite (y/n)?

You will get this message if you have already configured a public-private key for a separate server. To avoid this message I recommend setting up separate keys per this post.

At the prompt type:

/home/<client_user>/.ssh/id_rsa_<server_user>

Copy the new key to the server:

ssh-copy-id -i ~/.ssh/id_rsa_<server_user> <server_user>@<server_IP>

During the creation of your keys you may be prompted to set up a passphrase. I recommend using a passphrase if you are going to be using SSH to manage another machine remotely via the Terminal. The passphrase is an added layer of security. If you plan on using SSH as part of another service such as rsync you may want to not set a passphrase if your other scripts run in the background. I will leave this up to you and your Information Security folks to hash out. Bu-dum — bum.

Configure the new SSH Key on the Client

touch ~/.ssh/config
cd home/<client_user>/.ssh/
nano config

Once in the text editor, add the following:

Initial SSH configuration

Hit cntl-o to save, cntl-x to exit.

Note: If you plan on connecting to additional hosts, put their lines below the first configuration in the file as follows:

SSH configuration with additional server

Login to Raspberry Pi using SSH Public Key

# Modified YYYY-MM-DD by <author>.
#PasswordAuthentication yes
PasswordAuthentication no

As usual, hit cntl-o to save, cntl-x to exit. Remember to restart ssh service after editing the daemon config:

sudo systemctl restart ssh.service

You can confirm that the change has taken effect by logging out of SSH then attempting to log back in on the client machine. One of two things should happen: you are logged in thanks to your public key, or, the system asks you to enter your passphrase (assuming you created one when you created your public-private key-pair). If you get prompted to provide your passphrase you can hit enter a couple of times to see if SSH then asks for a password. If it does not then you have confirmed that the public key is the approved method of authentication.

PublicKey Troubleshooting

  • Delete authorized_keys on the server, they are located in the /.ssh folder.
  • If you are using Ubuntu, delete your OpenSSH keys from the Ubuntu Passwords and Keys app on your client. I beleive this program goes by the name of Seahorse when invoked from the command line.
  • Delete the /.ssh folder on your client for both your user as well as root accounts, elevate to root using sudo su, then rm -rf /root/.ssh Be very careful with this command as it will delete other things on your system willy-nilly if not used correctly!
  • Turn your client off and back on again to purge any cached keys stored in environment variables. Yes. I just told you to turn it off and back on again.
  • Change the hostname on your client temporarily to help troubleshoot stale keys. Inspecting authorized_keys with a text editor to ensure all keys with the old hostname were removed.
  • Use the following command to ensure stale keys are no longer present: ssh-add -l
  • If you see any keys as well as your old hostname after running the above command, it is a good indication that there are stale keys somewhere on your system, go back and repeat the above steps to purge them.
  • Re-generate keys for the server and client, use nil passphrase by hitting enter when prompted.
  • Use ssh-add -l to confirm the newly-generated keys as well as ensure that both the username and hostname are correct.
  • If all else fails go and grab a nice beverage then try again tomorrow.

Special Bonus: Interactive SSH

Open a new SSH session as follows:

ssh -X pi@<pi_IP>

You should now be able to type in the names of programs on your server and hit enter. If all goes well the program should open as a new window on your client. Be patient particularly if you are connecting via WiFi!

Summary

Server

Install openssh-server via:

sudo apt install openssh-server

Confirm SSH is running:

sudo service ssh status

Edit the SSH daemon configuration:

sudo nano /etc/ssh/sshd_config

Un-comment PubkeyAuthentication:

# Modified YYYY-MM-DD by <author>.
PubkeyAuthentication yes

Uncomment PasswordAuthentication:

# Modified YYYY-MM-DD by <author>.
PasswordAuthentication yes

Add an AllowUsers line directly below PasswordAuthentication as follows:

# Modified YYYY-MM-DD by <author>.
AllowUsers pi

Comment-out AllowGroups if it is present:

# Changed on YYYY-MM-DD by <author>.
# AllowGroups …

Save by hitting cntl-o then cntl-x to exit then restart your SSH daemon:

sudo systemctl restart ssh.service

Attempt an initial SSH connection from the terminal on your Pi:

ssh -v pi@<pi_IP>

You should now be logged into an SSH session on your Pi. Log-out and then proceed with setting up PublicKey access.

Go and check with your System Administrator to confirm if they are okay with you deleting the SSH keys before proceeding with these next steps!

Navigate to the hidden folder on your Pi containing the SSH keys:

cd .ssh

Edit the authorized_keys file:

sudo nano authorized_keys

Remove the entries from the file then hit cntl-o to save and cntl-x to exit. Alternatively, delete the entire file via:

rm /home/<client_user>/.ssh/authorized_keys

Client

rm -rf /home/<client_user>/.ssh

Generate a new set of SSH keys:

ssh-keygen

Enter a passphrase as appropriate then rename the new key file along the lines of:

/home/<client_user>/.ssh/id_rsa_<server_user>

Copy the new key to the server:

ssh-copy-id -i ~/.ssh/id_rsa_<server_user> <server_user>@<server_IP>

Ceate and edit a config file to enable SSH to point to the newly-created key:

touch ~/.ssh/config
cd home/<client_user>/.ssh/
nano config

Once in the text editor, add the following:

SSH configuration

Hit cntl-o to save, cntl-x to exit.

Connect to Server

Login to your Pi from the client:

ssh -v pi@<pi_IP>

You should be asked for your passphrase assuming you set one up.

Edit the config file:

sudo nano /etc/ssh/sshd_config

Change the PasswordAuthentication line is changed from yes to no, otherwise you may still be able to login to your server using a password:

# Modified YYYY-MM-DD by <author>.
#PasswordAuthentication yes
PasswordAuthentication no

Hit cntl-o to save, cntl-x to exit then restart your SSH daemon:

sudo systemctl restart ssh.service

Logout and login again from the client to confirm PublicKey still works.

Postscript

There are many more things you can do to improve the connectivity between your Raspberry Pi and another computer, for instance please check out this excellent Medium article on connecting an RPi to a Mac with an Ethernet cable. You may need to adjust your DNS settings once you perform this connection, more on this in another post!

What steps have you taken to set up SSH on your Raspberry Pi?

The Startup

Medium's largest active publication, followed by +756K people. Follow to join our community.

Eric North

Written by

Engineer with a small dash of quirkiness, always on the lookout for new things to learn. Avid sailor and adventurer. Publisher of good(?) ideas.

The Startup

Medium's largest active publication, followed by +756K people. Follow to join our community.

Eric North

Written by

Engineer with a small dash of quirkiness, always on the lookout for new things to learn. Avid sailor and adventurer. Publisher of good(?) ideas.

The Startup

Medium's largest active publication, followed by +756K people. Follow to join our community.

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store