[Part 2] How to setup your own environment for deep learning — For remote access
[This post was originally published here. To get a better formatting of the article go directly there.]
In part 1 we learned how to setup your computer locally for deep learning.
Now imagine you’re like me and you usually work on your laptop but you still want to use your computer with a powerful GPU whenever you want from anywhere in the world, in the same way as you would use Floydhub or Amazon AWS.
Wouldn’t it be great if you could run Pycharm or Jupyter on your laptop but with your code being executed on your distant powerful computer?
In this tutorial we will learn how to do this setup with security in mind.
I will refer to computer A as the computer with your setup from part 1 and computer B as the one which you want to access.
Audience
Like part 1 this tutorial is targeting both types of audiences: One with a basic computer science background and the other which don’t come from a similar background.
Since both these audiences are distinct, some basic parts of this article are between separators, mainly for the second audience.
Configuring your ssh access
For this step I’ll assume that your computer A and computer B are connected to the same network.
We start by configuring the ssh access so that you can log into your computer more securely than using a username/password combination.
What is ssh? For what purpose?
SSH or Secure Shell is a communication protocol which allows two entities to communicate through a secure channel (in our case the communication between computer A and computer B). To do this we will create a couple of asymmetric keys, a private key and a public key (practically it’s just two files containing weird numbers and letters). The private key should be owned only by you, you should never distribute it. As for the public key you can give it to whoever you want. In our case we will put the public key onto computer A and we will secretly keep our private key on computer B.
The private key can access and decrypt ressources encrypted by the public key, but the public key can only encrypt things, it can never decrypt them. Hence it is important to keep your private key secret. For more details, follow this link.
On computer B:
Go into your terminal and type this command:
ssh-keygen -b 4096The -b 4096 parameter lets you use a more robust combination of keys. You will be asked where to save the keys, press enter for the default location. When asked, choose a passphrase or don't use any by pressing enter (in case you get your private stolen you will be the only one to know the passphrase to access it).
At the end you should have a result similar to this:
Your identification has been saved in /home/ekami/.ssh/id_rsa.
Your public key has been saved in /home/ekami/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:D9OZWG28z9Hs7+kP+xr8Ehs69/fxPeBRyIJykDK64nI ekami@ekami-Desktop
The key's randomart image is:
+---[RSA 4096]----+
| . |
| o o o |
| . o . o = . |
| . . * = + + |
| . S + o o o|
| . . + *oo |
|. . . ..O=.|
|..E o.+*B|
|.. o+O/|
+----[SHA256]-----+Now go into the ssh directory with:
cd ~/.sshIn this directory you should find 2 files if you type the command ls, one called id_rsa, this is your private key and one called id_rsa.pub, this is your public key.
On computer A:
Now lets configure the ssh access so that you can connect to it from your computer B. You first have to download OpenSSH-server and let it run in background to allow secure ssh connection to come in.
On Ubuntu install it with:
sudo apt install openssh-serverNow open its configuration file with:
sudo nano /etc/ssh/sshd_configLocate the following lines in the file and change them so that they look as below (PS: The ...in the code only represents other text preceding / following the code):
...
RSAAuthentication yes
PubkeyAuthentication yes
AuthorizedKeysFile %h/.ssh/authorized_keys
...
# Change to no to disable tunnelled clear text passwords
PasswordAuthentication no
...With the nano editor save the file by pressing ctrl + X then answer Y and press enter.
Finally reboot the ssh service with:
sudo service ssh restart && sudo service sshd restartNow you need to get the id_rsa.pub (your public key) that you have generated on computer B. Use a usb key or something to transfer it to computer A then place it in the directory ~/.ssh of your computer A .
Remember the
AuthorizedKeysFile %h/.ssh/authorized_keysline from sshd_config? Good, now you may have guessed we will rename our id_rsa.pub to authorized_keyswith:
cd ~/.ssh && mv id_rsa.pub authorized_keysFinally we have to give it the proper rights with:
chmod 600 ~/.ssh/authorized_keysNow we just have to get the ip address of computer A with:
ifconfigIn my case I get:
enp0s3 Link encap:Ethernet HWaddr 08:00:27:97:d7:50
inet addr:192.168.1.26 Bcast:192.168.1.255 Mask:255.255.255.0
inet6 addr: fe80::23a:bfd4:7679:eb8/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:6326 errors:0 dropped:0 overruns:0 frame:0
TX packets:961 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:5076640 (5.0 MB) TX bytes:105266 (105.2 KB)Here my ip address is: 192.168.1.26 on my enp0s3 network interface.
Now the fun part, we'll try to connect to your computer A from computer B !
On computer B:
Open a terminal and type:
ssh your_computer_A_username@your_computer_A_ip_addressReplace your_computer_A_username by the username of your computer A and your_computer_A_ip_address by your the ip address you just retrieved from computer A.
In my case it looks like this:
ssh ekami@192.168.1.26If you now get something similar to this:
~/.ssh ➜ ssh ekami@192.168.1.26
The authenticity of host '192.168.1.26 (192.168.1.26)' can't be established.
ECDSA key fingerprint is SHA256:/3lt9qH4Pyo71wtXvgR4W9cvmR2ywAuK4F7/ivvoAWs.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.1.26' (ECDSA) to the list of known hosts.
Welcome to Ubuntu 16.04.2 LTS (GNU/Linux 4.8.0-36-generic x86_64)
* Documentation: https://help.ubuntu.com
* Management: https://landscape.canonical.com
* Support: https://ubuntu.com/advantage
82 packages can be updated.
60 updates are security updates.
The programs included with the Ubuntu system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.
Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
applicable law.
ekami@ekami-Desktop:~$Then congratulation, you’re connected to computer A and done with ssh!
Configuring Jupyter for remote access
Now lets secure our Jupyter notebook!
On computer A:
First you’ll have to create a .pem file which will be used to secure your internet traffic between your 2 computers with the https protocol instead of http.
Why https instead of http?
http is the default web protocol when you browse the internet, however it does not encrypt your communications. For instance, if you were to type your login/password on a website, this information would travel in plain text. Anyone having access to the same network as you can intercept them with the proper tools. https encrypts the communications and if someone is trying to intercept the communication, he will have to decrypt your information first!
To generate a .pem file enter the following command:
openssl req -x509 -nodes -days 365 -newkey rsa:1024 -keyout ~/.jupyter/mykey.key -out ~/.jupyter/mycert.pemYou’ll be asked some questions, answer them but if you’re not sure about the responses don’t worry it will still work for our purpose.
Now chose a password and enter it when prompted by this command:
python -c "from notebook.auth import passwd; print(passwd())"You’ll get a result looking like this:
sha1:cc4c9132d1c2:180655940d2472f34d1bf4f44d42564ef6791c42which correspond to your hashed sha1 password.
Now create the jupyter configuration file with:
jupyter notebook --generate-configAnd open it with:
nano ~/.jupyter/jupyter_notebook_config.pyAs before, find and replace the following lines (be careful to use your username in place of your_username and use your sha1 password you generated earlier instead of the one shown below):
...
## The full path to an SSL/TLS certificate file.
c.NotebookApp.certfile = '/home/your_username/.jupyter/mycert.pem'
...
## The IP address the notebook server will listen on.
c.NotebookApp.ip = '*'
...
## The full path to a private key file for usage with SSL/TLS.
c.NotebookApp.keyfile = '/home/your_username/.jupyter/mykey.key'
...
c.NotebookApp.open_browser = False
...
## Hashed password to use for web authentication.
#
# To generate, type in a python/IPython shell:
#
# from notebook.auth import passwd; passwd()
#
# The string should be of the form type:salt:hashed-password.
c.NotebookApp.password = 'sha1:cc4c9132d1c2:180655940d2472f34d1bf4f44d42564ef6791c42'
...Now you’re done with the jupyter configuration. Run it with:
jupyter notebookOn computer B:
Reuse the ip address of your computer A to access the jupyter notebook from a web browser of computer B. In my case it looks like this:
*Notice the https://
At this point you’ll probably get a security warning as shown above — don’t worry, this is perfectly normal. You can proceed to your notebook safely (in Chrome just click on the ADVANCED button then click on Proceed to 192.168.1.26 (unsafe)).
Why do I get this warning? Is my connection really secure?
It is — through the process of SSL certificate delivery. Basically there are people in this world called the certificate authorities who delivers the certificate we just created (the .pem and .key files). These authorities validate the certificates that we can trust. But since we created our own certificate and didn't tell them to trust us, we got this additional warning (you can see a list of trusted certificate Authorities from Chrome in Settings > Show advanced settings > HTTPS/SSL > Authorities). If you want more information take a look at this video.
And if you're wondering if your connection is really secure, yes it is as we now use the https protocol with all its benefits as explained above.
Once you passed this webpage enter your plain text password into jupyter to access it.
Congratulation, your jupyter notebook is finally configured for remote access!
Configuring the static IP
On computer A:
We now need to expose your machine to the internet so that you can access it from anywhere.
To do so we’ll first set a static IP on your computer A
Why a static IP address?
When you start your machine and connect it to your router, the latter will attribute it a random local IP address with a protocol called the DHCP. This is fine by default, but in our case we don’t want to deal with a new address each time we connect to computer A from computer B. Instead we would like it to be static so we will force our router to give us the one we tell him to.
To start off we’ll first need to figure out the network part of your local IP address and its host part.
Remember the ifconfig command we typed earlier?
Type it again and look for this line:
inet addr:192.168.1.26 Bcast:192.168.1.255 Mask:255.255.255.0The Mask allow us to figure out the network and the node part of our IP address. Here as the Mask is 255.255.255.0, our network part will be 192.168.1 and our host part will be .26. If the mask was instead 255.255.0.0 then our network part would be 192.168 and our host part would be .1.26.
Now we will create a custom crafted IP address by keeping the network part but by changing its host part. I chose 170 here so my local ip address will looks like this in the future: 192.168.1.170.
Lastly we need to get the address of your gateway (the router in your case) with this command:
route -nIn my case it shows:
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 192.168.1.1 0.0.0.0 UG 100 0 0 enp0s3
169.254.0.0 0.0.0.0 255.255.0.0 U 1000 0 0 enp0s3
192.168.1.0 0.0.0.0 255.255.255.0 U 100 0 0 enp0s3So my gateway address is 192.168.1.1.
Now to force the router to use our custom crafted ip open this file with nano:
sudo nano /etc/network/interfacesThe content should look like this:
# interfaces(5) file used by ifup(8) and ifdown(8)
auto lo
iface lo inet loopbackReplace it with:
# interfaces(5) file used by ifup(8) and ifdown(8)
auto enp0s3
iface lo inet loopback
iface enp0s3 inet static
address 192.168.1.170
netmask 255.255.255.0
gateway 192.168.1.1
dns-nameservers 8.8.4.4 8.8.8.8Replace enp0s3 by the name of your network interface (it's the first thing which appear when you use the ifconfig command), replace 192.168.1.170 by your custom crafted IP, the netmask by the Mask address of the ifconfig command and gateway by your gateway IP.
Finally restart your network interface with:
sudo service network-manager restartIf you lose your internet connection with the command above, reboot your computer A. If after reboot you still don’t have any internet connection, try choosing a different custom crafted IP address and repeat the process.
Now if you type:
ifconfigYou should have an output similar to this:
enp0s3 Link encap:Ethernet HWaddr 08:00:27:97:d7:50
inet addr:192.168.1.170 Bcast:192.168.1.255 Mask:255.255.255.0
inet6 addr: fe80::a00:27ff:fe97:d750/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:62 errors:0 dropped:0 overruns:0 frame:0
TX packets:46 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:5160 (5.1 KB) TX bytes:5426 (5.4 KB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:52 errors:0 dropped:0 overruns:0 frame:0
TX packets:52 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1
RX bytes:3888 (3.8 KB) TX bytes:3888 (3.8 KB)Congratulations your machine now has a static IP address!
Configuring the Port forwarding
Now you have a static local ip address, it’s great but not enough for your machine to be visible on the internet. We now need to configure the port forwarding.
What is the Port forwarding?
Your router act as a gateway between the internet and your local network. Hence you need to tell your router (gateway) where to look when you ask it for resources from computer A.
Port forwarding enables that. Every service communicates through ports. For example, when you access your Jupyter notebook from computer A, you type https://127.0.0.1:8888 in your browser: 127.0.0.1 is the local ip address and 8888 is the port used for the Jupyter service. Same goes for ssh, ssh uses the port 22 by default. For instance, consider the following picture:
As you can see it will be difficult for your router to know where is the Jupyter service if you don’t tell it to connect on computer A.
This part will be a bit tricky as it dependent of your router interface. Here I’ll give examples based on mine (an ASUS RT-AC87U) but you should still be able to replicate these instructions on your side. Most of the routers web interface can be accessed by typing the gateway address in your browser.
In my case it’s 192.168.1.1 and I get something like this:
Enter into your router configuration and look for WAN and/or Port Forwarding.
Then add to the list the following mapping:
Replace 192.168.1.170 by your own local IP address then click "Apply".
At this stage you should be able to access to your machine from the internet. Lets test this!
On computer A:
Start your jupyter notebook with:
jupyter notebookNow we’ll try to access it through your external ip address, the one visible by anyone on the internet.
Go to this website to get it and type in your browser https://your_ip_address:8888, in my case it would be https://81.64.208.179:8888.
If you have access to jupyter congratulations! You can type the same address from any device you have from any network (try to access it from your 4G connection!).
Configuring the Dynamic DNS
You may be tempted to feast already but hold on, we still have some issues to consider first.
One of them is about DDNS, we have to create a domain to map your computer A external IP address to a name in case your IP changes.
What is DNS/DDNS? Why do I need it?
DNS or Domain Name system is a naming system used to identify resources. In our case we want to give a name to our external IP address.
For example when you type https://google.com in your browser what happen is that a request is sent to one or more DNS server(s) to translate it to an IP address, then and only then you can access to google.
So why do we need this? Can't we just use our ip address to reach computer A? Yes we can but there is a little kink here.
Most ISPs (Internet service providers) give you dynamic IP address, which means that at some point, your external IP address will change. Imagine you're on a trip to French Polynesia, you try to connect to computer A located in France but it doesn't work because its IP has changed. What can you do? Well... not much, that's why having a fixed Dynamic DNS which update itself when your IP changes is essential.
If you want to know more about how DNS works check out this video.
There is a lot of ways to register dynamic dns but here we’ll use the No-ip service. Register yourself on the website then go to your dashboard, select “Dynamic DNS” and add yourself a dynamic dns, in my case it looks like this:
You’ll see a little notice under your ddns: “No dynamic update detected”. This is because your router didn’t tell the no-ip service where it is. To do so you’ll find a tab on the left menu named “Dynamic update client”. Click on it and download the client.
Now open a terminal in the directory where you extracted the client and install it with:
make && sudo make installEnter your credentials, keep the 30sec interval and answer “No” at running something at successful update.
Now download this script and execute the following command in the repertory where you downloaded the script:
chmod +x noip2_startup.sh && sudo mv noip2_startup.sh /etc/init.d/ && sudo chown root:root /etc/init.d/noip2_startup.shYou also need your service script to execute on startup with:
sudo update-rc.d noip2_startup.sh defaultsFinally reboot your computer. If you now go to your no-ip dashboard the “No dynamic update detected” message should have disappeared.
Great you’re all set and if you didn’t shut down your jupyter instance you should be able to access it with your domain name. In my case it would be https://ekami1.ddns.net:8888. You can also try to log in with ssh.
Note:
With the technique we used here your computer A must be powered on when your IP changes. But as you never really know when it could happen then you must not power off your computer A for too long. Other techniques to setup more robust ddns (directly on the router or by running the client on a raspberry pi for example) exists but are not covered here.
Start/stop your computer remotely (Wake on lan/AC back)
On computer A:
Now imagine you’re still enjoying your trip in French Polynesia but your computer A abruptly shut down because of a tiny loss of current. Or worse it get stuck in a loop and you have to force reboot it, what can you do? Well, at this stage… not much, but don’t lose hope, solutions exists!
The first one is called Wol or Wake on lan, which allow you to wake up your computer with your network card. I cannot enter into the details of the configuration of this here as it is specific to your motherboard. Refer its manual to learn how to activate it. For me it is a simple entry in the bios.
Still, there is 2 drawbacks with Wol:
- You cannot force shut down your computer
- In some cases it won’t work if you didn’t shut down your computer properly
Beware of the last point. I already found myself not being able to wake up my computer because I used a force shut down previously… Now there is a little hack to achieve our goal without spending a lot of money. I call it the Smartplug! You can find one on amazon for ~ $30 and remotely cut/restore the current to your computer remotely.
Thing is that you also need to activate an option in your computer BIOS called “AC back” (for your motherboard it may have another name). What AC back permit is to start your computer as soon as it detect the current coming back to your computer. From my motherboard manual:
AC BACK Determines the state of the system after the return of power from an AC power loss.
Always Off: The system stays off upon the return of the AC power. (Default)
Always On: The system is turned on upon the return of the AC power.
In my case I just use the “Always On” option so each time I want to reboot I just power off/on my smart plug, the current turns on my computer automatically, boot on ubuntu and voilà!
If you’re not satisfied by these solutions or you’re looking for more challenges you can still create yourself a robotic arm to click on your power button.
Configure Pycharm for remote execution
[/!\ Remote execution is only supported in the professional edition of Pycharm]
Now the last part, if you’ve come this far then I must admit, you’re a warrior! Let’s get started!
On computer B:
Launch Pycharm and go to File > Default Settings > Project Interpreter then click on the little gear and "Add remote".
Add your credentials as I did in the picture below:
Notice the python interpreter is our conda virtual env we created on Part 1.
Now go on File > Settings > Build, Execution, Deployment > Deployment (notice we are now in the project settings, in macOS the breadcrumb is PyCharm > Preferences... > Build, Execution, Deployment > Deployment) and click on the "+" sign to create an SFTP connection, give it a name and fill the instructions according to what I have below:
Go to the Mappings tab and remove the one existing until you have only one left, then fill the fields as below and click on “Use this server as default”:
Now click “Apply” and from the Deployment menu click on “Options” and change “Upload changed files automatically to the default server” to “Always” then click “Ok”.
Finally go on edit configuration:
Add a Python configuration and fill the fields as below:
Click “OK” , make a change to your script, save it (it will be automatically uploaded to the computer A) and execute it with the green play button.
Congratulations! This concludes the two part series on how to setup your own deep learning rig. All the best for playing around in the world of neural nets, CNNs and RNNs! :D
Acknowledgements
If you’ve come this far then you’re really tenacious!
Thanks for supporting me and thanks to Akshay Lakhi for his review. I hope you enjoyed your reading.
Originally published at tuatini.me on March 19, 2017.
