Setting up a Deep learning machine in a lazy yet quick way

How about do it today?

November of last year, I took the “Deep learning for coders — Part 1” course and since then I have been trying out Deep networks for a variety of problems areas like Computer vision and NLP among others. I got a p2 EC2 instance ($0.90 / hr for 1 K80 GPU) and was running all my experiments there. In the process, I spent way more than I was expecting on AWS.

If I was very vigilant in using my instance just when I am running on GPU, I would have saved good amount of money. But that was not always possible without compromising on productivity or flexibility. Also K80s are not that great and realized I should invest in a better GPU to accelerate experimentation and reduce overall costs.

Although I knew I wanted to upgrade and started researching about graphics cards, parts and configurations, it took me a while before I have this beauty setup in our home.

There are some good GTX 10 series laptops, but I decided to go with a desktop for the sake of upgradability. And I am super glad I did, as there is a new card GTX 1080Ti which is released exactly a week after I setup this system. So now, I can add that as my second card :-)

So if you are in the position I was where you are on the edge of building your own machine but a little unsure of time that you need to invest in getting that setup, this post is for you. And this is inspired by my fellow students, mainly Yad Faeq, Brendan Fortuner and our professor Jeremy Howard.

Instructions

Brendan Fortuner, my classmate has written a very accessible and excellent blog post on building your own Deep learning machine which includes 6 key steps as outlined to the left. So stop here and skim through it now, you will need to go back to that blog post when you are doing the actual setup. I will only be covering some additional perspectives here.

Some tips to speed up the process in a lazy way :-)

On choosing parts

Start from Brendan’s list, trust me, it’s good. Some things to keep in mind when you customize the list based on your budget.

  1. Spend most of your money on GPU in this order GTX 1080Ti -> GTX1080 -> GTX1070
  2. Get good RAM, I got 32GB and more room to expand
  3. Get SSD, accounts for so much productivity boost.
  4. Get a motherboard which supports multiple GPUs, so that you can upgrade in future.

Hardware assembly

You have two options here:

Our toddler “helping out” with the setup
  1. Get your system assembled at a place like Central computers for like 80$, they also do a burn test which can save you some pain if your parts come broken. I used this service as I figured assembling it myself with a toddler at home would be a lot more fun than I would want it to be. The service was great and on time. Got great recommendations for motherboards which support multiple graphic cards, PSU required for your current and future parts. They install required drivers. They even install Windows, but I wanted Ubuntu so I did that part myself. A bonus, I liked their case selection :-)
  2. If you are on a tight budget, buy a used Dell workstation. Order a GPU online and you can plug it in yourself.

Setup workflow and test within your LAN

Src: https://www.qnap.com/images/products/Application/TS109AP05.htm

Assuming you would want to not always work from home and would like to access your machine remotely from your laptop, you will need to setup remote workflow. This picture gives a brief overview of WAN and LAN and what needs to happen. But first, I would recommend setting up your workflow within LAN to test everything works within LAN.

  1. Make sure you are able to ssh within your LAN.
  2. I mostly use terminal and Jupyter notebooks for coding. If you have a simialar workflow, make sure you are able to run Jupyter workflow from your laptop
Server:  jupyter notebook — no-browser — port=8888
Laptop: Put this in your ~/.bash_profile using your ACCOUNT@LAN_IP
___________________
desktop_ip=”ACCOUNT@LAN_IP”
remote_jupyter() {
ssh -NL $1:localhost:$1 $desktop_ip
}
___________________
Laptop: $remote_jupyter 8888

Password-less ssh

Although this step is not required, I would hate to type my long password every time I connect, and also I have better trust on strong SSH keys than my passwords.

Follow this if you already have ssh setup on your laptop.

a@A:~> ssh b@B mkdir -p .ssh
b@B's password:
a@A:~> cat .ssh/id_rsa.pub | ssh b@B 'cat >> .ssh/authorized_keys'
b@B's password:

If not follow instructions here to setup your keys. And disable password ssh login on the server for better security unless you plan to share your work station with others. Update the following parameters in your /etc/ssh/sshd_config

ChallengeResponseAuthentication no
PasswordAuthentication no
UsePAM no

and restart SSH deamon/etc/init.d/sshd restart

Reserve LAN IP in your DHCP

Internal IPs are also dynamically allocated, so to avoid IP conflict, reserve your internal IP in your DHCP settings, which would essentially bind one of your LAN IPs to the MAC of your machine. See bullet 2 below for how to do it.

Make it work from outside of LAN

  1. Open ssh port for access from outside LAN : We use TP-Link router at home, its Admin page is at 192.168.0.1 with default credentials: admin/admin. You can check instructions for your specific router here. Once you get to the admin console, they all look pretty similar. All that you need is to open up port 22 for ssh.
  2. Address Reservation: DHCP -> Address reservation and bind MAC to an internal IP
  3. Update LAN_IP with WAN_IP in your ~/.bash_profile . To get your WAN_IP do $dig +short myip.opendns.com @resolver1.opendns.com

Finally, test your work flow from outside of LAN. Simple way is to connect your laptop to your phone hotspot (so that it is not on LAN), and test the workflow. And my models train in half the time now with just one GTX 1080 compared to AWS p2 instance. Yayy!

So far, my WAN IP has not changed, so I decided to not worry about static IP yet. Especially since I do not have any production models deployed on my home machine and only occasionally use it remotely. Although I think, I will look into static ip options like noip if needed.

Would love to hear if you have setup a similar machine. And please share your favorite productivity tips in these kind of setups.