Using Tensorflow and Pytorch in Pycharm on Windows 10

Pytorch installation on Windows is a pain and Tensorflow isn’t available on Python 2.7 for windows which ensues in a nice segue to the solution…

You can use this blog post either as a reference guide to reinstall your bash and expiate changing permissions for ssh host keys and messing with chmod on the command line like I did ,or as a starting point to put aside dogma and try something new and interesting.

Going on a wild goose chase to reinstall modules or packages and restore everything by making a myriad of setting changes, gets me super nettled. It’s cumbersome and tedious, so to save my time and sanity, lest it should happen in the future again, I’ve gathered a cornucopia of commands and install guidelines to ensure a clean and successful workspace for Machine Learning and Deep Learning code.

Why Pycharm?

A good IDE is conducive to efficient and effective coding practices. One of the best IDE for all things Python related, that I have come across, is Pycharm. If you haven’t used it yet, I implore you to try it and I guarantee that you’ll never go back. Try getting the Professional Edition (maybe for a year) released under a proprietary license as this has extra features that will come in handy later. It is cross-platform, with Windows, macOS and Linux versions. Developed by JetBrains, it has good integration with Version Control systems and provides code analysis as well as unit-testing capabilities.

Pycharm has helped me extirpate so many bugs that I never would have been able to pinpoint if not for the Inline Debugger for easy inspection of variables, function parameters and objects right inside the editor. It’s very useful when you are working with large code bases or if you want to switch between different python versions or environments for your project interpreter.

Why Linux?

If you want to go through why most people prefer Linux, you can read the following link

Here comes the segue….

What is Bash on Ubuntu on Windows?

“A linux on windows without a vmware or virtual box, that’s heresy !!”

WSL is an acronym for Windows Subsystem for Linux. Colloquially called Bash_on_Ubuntu_on_Windows, this is more than just a Bash shell. It’s a full compatibility layer for running Linux applications on Windows. Microsoft teamed up with Canonical, aka the creators of Ubuntu Linux, to build this new robust infrastructure within Windows. You’ll find several links online to configure your developer mode settings and install ubuntu on your windows 10.

Some of the reasons why I prefer WSL to Virtual Box is that

1) You can access Windows Files in Bash, and Bash Files in Windows. You get the flexibility to work on the same file in Linux and Windows. When you’re in the Linux environment, you can access your Windows drives from under the /mnt/ folder. Your C: drive mapped to /mnt/c and your D: drive is located at /mnt/d,and so on.

2) You can also write a Bash shell script on Windows and run it. Your Bash script can access your Windows files stored under the /mnt folder, so you can use Linux commands and scripts to work on your normal Windows files. You can use the “bash -c” command to directly execute a Linux command. Bash will simply run the command and then quit. To run a Windows program, type the path to an .exe file and press Enter.

3) Rather than using Cygwin or the current beta of the OpenSSH implementation for Windows, we now can run SSH natively via the ssh command and also copy files securely over SCP using the built-in scp command. Just ssh into the server and you can manage it from the terminal. Running remote scripts works quite well

4) Ease of using git from command line

5) You don’t have the overhead of a VM, so programs hypothetically should run faster. No need of dual booting. You also don’t need to reserve memory for a VM that it might not need; you’re running programs as Windows processes, so they use the same memory pool that any other process on the system would.

6) you can’t edit/open/delete Windows apps or alter Windows system settings which serves as a great failsafe, as we all know mistakes can happen when using the command line

7) Easy install, uninstall or update of the Linux system with the lxrun commands

It also has a growing community that you can keep up with on

I’m not trying to pander but while some censure it for being stringent with released features that lack few things like an impending CUDA support, which in my opinion should have been on their priority list, it’s a really cool feature that Microsoft came up with as an answer to the plight of developers who haven’t made a complete switch to a Linux but wish to use those features.


Installing Tensorflow and Pytorch on WSL:

Tensorflow

For python 2.7

curl -O https://repo.continuum.io/miniconda/Miniconda2-latest-Linux-x86_64.sh
chmod a+x Miniconda2-latest-Linux-x86_64.sh
./Miniconda2-latest-Linux-x86_64.sh
rm Miniconda2-latest-Linux-x86_64.sh
source ~/.bashrc
conda upgrade — all
conda install anaconda-client
conda env create suhangpro/tf-py2

To update to the current version

pip install --ignore-installed --upgrade https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-1.2.1-cp27-none-linux_x86_64.whl

For python 3.6

curl -O https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh
chmod a+x Miniconda3-latest-Linux-x86_64.sh
./Miniconda3-latest-Linux-x86_64.sh
rm Miniconda3-latest-Linux-x86_64.sh
source ~/.bashrc
conda upgrade — all
conda install anaconda-client
conda env create suhangpro/tf

To update to the current version

pip install --ignore-installed --upgrade https://storage.googleapis.com/tensorflow/windows/cpu/tensorflow-1.2.1-cp35-cp35m-win_amd64.whl

If you have issues detecting conda, ie., Module not found error, you just need to set the path so that it can find the miniconda installation of python.

export PATH=”/<enter path till>/miniconda2/bin:$PATH”
export PATH=”/<enter path till>/miniconda3/bin:$PATH”

Pytorch

You can find the conda package installer commands from

conda create -n pytorch-py2
conda install pytorch torchvision -c soumith

Jupyter notebook

If you are unable to use jupyter notebook within an environment, you can try

conda install jupyter

within the conda environment

printf “set KMP_AFFINITY=disabled\nexport KMP_AFFINITY” >> ~/.bashrc

and then use the following hacks

pip uninstall pyzmq
sudo add-apt-repository ppa:aseering/wsl
sudo apt-get update
sudo apt-get install libzmq3
conda install -c jzuhone zeromq=4.1.dev0
sudo apt-get install python-zmq

Jupyter and ipython notebook support hasn’t been made available on remote interpreter so by all means try to enjoin and instigate by voting for the related issue

https://youtrack.jetbrains.com/issue/PY-14605

to increase its priority and be notified about updates


Using Remote Interpreters in Pycharm :

In your bash make sure you run the following

sudo apt install -y build-essential
sudo apt install -y gdbserver
sudo apt install -y openssh-server

This will generate the sshd_config. If you feel something is wrong with your file, you can reinstall the openssh-server again

sudo vi /etc/ssh/sshd_config

Edit the entries: PasswordAuthentication to yes ,UserPrivilegeSeparation to no,Port address is 22 by default but you can change it if some other application is already bound to that. Otherwise just let it be.

sudo ssh-keygen -A

This command will generate the 4 keys .Now all you need to do is start the service

sudo service ssh start

check the status with

sudo service ssh status

which should tell you that sshd is running

sudo service ssh stop to stop

Verify by performing an ssh to your localhost. Open another bash terminal and type ssh localhost or ssh 127.0.0.1 and enter the password when prompted. This should let you know that your service is functioning properly

Having a working ssh service is sine qua non. This might seem, esoteric at first but trust me, it’s not that troublesome. I’ve noticed that closing the terminal has the propensity play possum to wreak havoc on your bash and make it unresponsive leaving you with only one option: restart your system.

Now open Pycharm> Settings> Project Interpreter > Select the Settings button next to the dropdown to Add Remote (available in the Professional Edition)

Choose the SSH_Credentials radio button and fill the corresponding details

You need to keep the service running in your bash terminal in the background and not close it.

This will connect you to your environment on Ubuntu and you’ll be able to see all the installed packages (including tensorflow or pytorch based on the environment you chose).

Now to be able to access your files and you need to set the mappings in the interpreter.

Debugging

Turns out the debug mode expedites your code execution using cython. Run the command prompted in the pycharm console to build, in your bash terminal. As stated by the developers, the debugger is 40% faster in the general case and over 130% faster when you have the Cython modules compiled and installed. On Windows, you don’t need to do anything — you will get these improvements automatically for Python 2.7, Python 3.4, and Python 3.5 interpreters. On OS X and Linux, you need to do a one-time compilation using any Python on your system matching the Python version you need, as shown in the screenshot above. For example, if you use Python 2.7 and Python 3.4, you need to run the Cython speedups with an interpreter matching those two versions. Each time you run it, a compiled speedup will be saved in your PyCharm application, for that Python version. We need to do this part since we use Ubuntu

One clap, two clap, three clap, forty?

By clapping more or less, you can signal to us which stories really stand out.