How we deployed Duo 2FA for SSH to 600 Ubiquiti servers

Reilly Chase
Oct 9, 2019 · 6 min read
Image for post
Image for post

HostiFi Update

I started HostiFi, a Ubiquiti cloud hosting service, in May 2018. In January 2019 I went full-time on it, and since then I’ve been sharing monthly updates on website traffic, revenue, and what I’ve been working on at https://rchase.com.

My last blog post about HostiFi was in May 2019, when I wrote “Why I’m completely rewriting my $5,735 MRR SaaS” which was about some major changes that I’m building, including a website rewrite to move off of WordPress to a Laravel Spark platform.

I estimated that rewrite would take 2–3 months, and it’s been 5 months now, but it is finally getting close to completion!

Here is a preview of the new homepage:

Image for post
Image for post

Here’s a quick update on progress since my last blog post before I write about the Duo deployment.

Image for post
Image for post
  • The business has doubled in size in several areas in the last 4 months— monthly recurring revenue increased from $5,735 to $11,403, Ubiquiti devices connected increased from 12,000 to 28,000, subscribers increased from 266 to 538, and servers increased from 309 to 600

Deploying Duo 2FA for SSH to 600 Ubiquiti servers

Image for post
Image for post

Since hiring Safwan to help with support, I’ve been thinking a ton about security as a team instead of just security as an individual. First off, we started using 1Password as a team (previously I used KeePass). 1Password manages storing passwords securely, reminds us to rotate them, and keeps separate credentials for every account to protect against password reuse.

In addition, we are 2FA’ing every account we can.

As I went about creating accounts in our various systems for Safwan, I’ve also followed the principle of least privilege by setting the account permissions so that he only has the access he needs to do his job and nothing extra.

One of the big steps in this process of securing things has been figuring out how to deploy Duo for 2FA SSH on all 600 of our Ubiquiti servers.

I’ve already been using 2FA in a way, by using public key authentication only, an attacker would need to have both the private key and also know the password to decrypt the key before being able to gain access to a server, but Duo adds the phone verification to the mix as well so it is more like 3FA instead of 2FA.

I was happy to find out that Duo is free for under 10 users

Image for post
Image for post

But I quickly upgraded to a paid plan when I realized the free plan does not allow you to create policies — for example, I wanted to enforce using the Duo app, and disable SMS as a 2FA option because it’s not secure. I upgraded to a paid plan which requires a minimum of 10 users at $30/month. No problem with the pricing though, this is a huge security benefit for us.

After purchasing, I was able to change the allowed 2FA methods

Image for post
Image for post

I enrolled some users, connecting the Duo app with each account

Image for post
Image for post

And created a UNIX Application for SSH 2FA

Image for post
Image for post

Next, I spun up a few servers on Vultr and began testing out the 2FA installation and integration process.

There were 2 methods available to integrate Duo, with documentation on each.

login_duo method

pam_duo method

I used the login_duo method because it was easier to set up multiple Duo users sharing a single root account, logging in with public keys instead of passwords, which is what we needed.

Once I had perfected the list of commands needed for the integration from testing manually on fresh servers, it was time to write a Python script to deploy all the commands to our actual 600 Ubiquiti servers.

I open-sourced that script here:

Here’s what the script does:

  • Pulls a list of customer server names from our database
Image for post
Image for post
  • Compares that list to a file “duo_already_done.txt” to see which servers don’t have Duo implemented yet
Image for post
Image for post
  • For each server that needs Duo implemented, it logs into it via SSH
Image for post
Image for post
  • It counts the number of SSH keys that already exist in /root/.ssh/authorized_keys to see if there are more than 2. If there are more than 2, it saves the server name to “duo_needs_fixed.txt” which we will later write another script to deal with. Basically, some customers currently have root access, and we are going to remove their root access now and instead give them a limited privilege user so they can still change Ubiquiti files but not server files.
Image for post
Image for post
  • The rest of the script executes the actual Duo install and integration commands
Image for post
Image for post

Closing thoughts

2FA, when implemented properly (no SMS!), is one of the best things we can do to add security to accounts. We also recently published a guide on how you can implement 2FA on your UniFi server here: https://support.hostifi.net/en/articles/3385881-how-to-setup-2fa-for-unifi

UNMS 2FA can be easily enabled as well under Settings > Users.

If you use UniFi or UNMS, check out our cloud hosting solution at https://hostifi.net! We provide updates, SSL install, monitoring, and support.

Image for post
Image for post

HostiFi

#UniFi and #UNMS cloud hosting

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store