Python Environments, Oh My!

June 22nd, 2020

(image credit: xkcd)

Okay okay, maybe its not quite that bad; but I did run into my first version of this craziness!

I ran into some brief difficulty trying to execute NumPy within PyCharm; I had up until this point not needed access to NumPy in my learning of introductory Python, and had been using virtual environments for each project I created. When trying to access NumPy from a virtual env, Pycharm insisted I did not have NumPy on my machine. I did in fact have it, installed as part of Anaconda.

I thought I could navigate the issue by switching my interpreter from a virtual environment to my system Python, which until recently had been 3.8. However, when I downloaded Anaconda, it also installed Python 3.7. Switching Pycharm to 3.8, unsurprisingly, still did not have any packages like NumPy available. Then I recalled that I had earlier made 3.7 my default Python per the Metis program’s recommendation. So, I switched Pycharm’s default interpreter to Python 3.7… and still no NumPy! I then tried what Pycharm had listed as my Anaconda environment Python as my interpreter, thinking surely NumPy must be in the same downloaded environment… to yet again no avail.

Ultimately, I looked up the location of my default python from terminal (it was ~opt/anaconda3/bin/python3.7), set that path as my default interpreter manually inside Pycharm (it was not offering this particular system Python as an option on its own), and then, finally, I could access the the NumPy modules I so desired. How convoluted!

This all comes from, like the image above suggests, using different package managers at different times — homebrew, pip, anaconda, etc. — to all install different things… I uninstalled Python 3.8 (both versions, as apparently I had 2 at different locations urs/local/bin/python3.8 and /Library/Frameworks/python.framework/Version/3.8/bin/python3 as well…) for simplicity’s sake, and now have 2 copies of 3.7 on my machine (the anaconda env listed above, and another local at /usr/local/bin/python3.7). I think this will keep things straight for now, and I *believe* these are the only versions of Python remaining on my machine, but, I see how easy it can be to get tangled.

Any more senior data scientists with recommendations on how to keep all this straight, or your advice on best practice, I’d love to hear from you!