SSH certificate management using AppViewX

7 min readJun 19, 2020

The SSH actually has the functionality to use a certificate authority to authenticate servers and clients. This works both ways, using this system we can authenticate a host to a client for avoiding confusing messages about being unable to validate the authenticity of the host. We can also validate the client to the host, allowing us to register a new SSH key in one place and allow access across your organization.

Solves the problem

This solves the problem of having to manually check and verify the host key fingerprint every time we connect to a new server. If the server key is signed by a CA that we choose to trust us then it will not be asked to verify the fingerprint.

How does AppViewX Automates SSH Certificate Management

· AppViewX will be CA server

· Select user or host certificate

· Select the device

· Fetch the public key from host/user machine

· Fill the cert parameters

· Sign the cert and push it to appropriate path of host /user machines

How does this works

Configure host certificates:

We will start by configuring certificates that will authenticate our servers to our clients. This will allow our clients to connect to our servers without needing to question the authenticity of the server.

We begin on the machine that we will be using as the certificate authority. In this example, we’ll refer to this as “gs-appviewx-dev29.apvxlab.com”.

Generating Signing Keys

First, we need to generate some RSA keys that will function as the signing keys. Use any user, but the root user is probably a good idea. We will be creating keys called “server_ca” and “server_ca.pub” since these will be used to authenticate our servers.

Let’s create these keys in our home directory:

It will be asked to create a passphrase. This will add an additional layer of protection to your key in the event that it falls into the wrong hands. Once this is finished, we will have a private and public key in your home directory:

Signing Host Keys:

Now that we have our keys, we can begin signing our host keys.

We should start by signing the host key of the certificate authority itself. We can do this using the following syntax:

ssh-keygen -s signing_key -I key_identifier -h -n host_name -V +52w host_rsa_key

Let’s go over what all of this means.

-s: This is the private key that we just created that we will use to sign all of the other keys.
-I: This is a name that is used to identify the certificate. It is used for logging purposes when the certificate is used for authentication.
-h: This marks the resulting certificate as a host key, as opposed to a client key.
-n: This is used to identify the name (user or host) that is associated with this certificate.
-V: This specifies how long the certificate is valid for. In this instance, we specify that the certificate will expire in one year (52 weeks).

Afterwards we specify the key that we want to sign.

In our case, to sign our own host RSA key, we will use a line that looks like this. We are going to identify this server as “host_server”. We will be prompted for the passphrase we used when creating the signing key:

[root@gs-appviewx-dev29 ~]# ssh-keygen -s server_ca -I host_server -h -n gs-appviewx-dev29.apvxlab.com -V +52w /etc/ssh/ssh_host_rsa_key.pub

Enter passphrase:

Signed host key /etc/ssh/ssh_host_rsa_key-cert.pub: id “host_server” serial 0 for gs-appviewx-dev29.apvxlab.com valid from 2020–05–19T14:16:00 to 2021–05–18T14:17:53

[root@gs-appviewx-dev29 ~]#

As we can see from the output, our certificate is valid for one year. It has been created in the same directory as our server host key (/etc/ssh/) and is called “ssh_host_rsa_key-cert.pub”.

Now that we have signed our host key on the certificate authority itself, we can sign the host key for the separate SSH server we’re trying to authenticate to clients.

Copy the host key from our SSH server. We’ll refer to this machine as “gs-appviewx-dev23.apvxlab.com” and we can do this using scp:

[root@gs-appviewx-dev29 ~]# scp root@gs-appviewx-dev23.apvxlab.com:/etc/ssh/ssh_host_rsa_key.pub .

root@gs-appviewx-dev23.apvxlab.com’s password:

ssh_host_rsa_key.pub 100% 382 275.1KB/s 00:00

[root@gs-appviewx-dev29 ~]#

Now, we can create a certificate from this file using the same method we used above. We’ll need to change some values to refer to the new host we’re signing:

[root@gs-appviewx-dev29 ~]# ssh-keygen -s server_ca -I client1 -h -n gs-appviewx-dev23.apvxlab.com -V +52w ssh_host_rsa_key.pub

Enter passphrase:

Signed host key ssh_host_rsa_key-cert.pub: id “client1” serial 0 for gs-appviewx-dev23.apvxlab.com valid from 2020–05–19T14:23:00 to 2021–05–18T14:24:20

[root@gs-appviewx-dev29 ~]#

Now, we need to copy the generated certificate file back onto the host. Again, we can use scp for this:

Afterwards, we can delete both the SSH server’s public key and certificate from our authentication server:

We now have the signed certificates in place, we just need to configure our components to use them.

Configuring Components to Use Host Certs:

First, we need to continue with both of our servers to make them aware of the certificate files we created.

On both of these machines, we’ll have to edit the main SSH daemon configuration file. Make sure you are editing the sshd_config file, not the ssh_config file:

If you can find a HostCertificate line, modify it. Otherwise, add this to the bottom of the file. We need to establish to path to our host certificate file:

Save and close the file when you are finished.

Now, restart the SSH daemon to make these changes happen:

Do this on all of the servers you are configuring host certificates for.

Now, our servers are configured to use the certificate, but our client does not know how to check the certificate that the server will present.

On our client machine, which we’ll be referring to as “client.example.com”, open or create the “~/.ssh/known_hosts” file:

We need to remove any entries that have to do with the servers we’re configuring for certificate entry. It may be best to delete everything.

Afterwards, we need to add a special entry that specifies the public key that we should use to check the certificate that our hosts will give us during login. Start it off with @cert-authority. Afterwards, it can include a domain restriction where the key will be applied, followed by the public certificate authority key that we’ve been signing everything with.

On your certificate authority machine, we can get the public certificate signing key by typing:

[root@gs-appviewx-dev29 ~]# cat ~/server_ca.pub

ssh-rsa y******************************************************************************************y iwdh root@gs-appviewx-dev29.apvxlab.com

[root@gs-appviewx-dev29 ~]#

Using this information, the line in ~/.ssh/known_hosts file should look like:

[root@gs-appviewx-dev23 ~]# cat ~/.ssh/known_hosts

@cert-authority *.apvxlab.com ssh-rsa A***********************************************************************************************************g root@gs-appviewx-dev29.apvxlab.com

[root@gs-appviewx-dev23 ~]#

Save and close the file when you’re done.

Now, when you visit the SSH server for the first time from ssh client (using the full host name), it should not be asked whether trust the remote host. This is because the host has presented its host certificate , signed by the certificate authority. We’ve checked our known hosts file and verified that the certificate is legitimate.

SSH Certificate authority:

gs-appviewx-dev29.apvxlab.com