How to set up a Virtual Environment for Jupyter Notebook With Pip And Conda (Very Simple Guide).

I don’t like making long introductions. If you do not know what virtual environments are and their utility, I put at the very end a short intro about them.

Let’s remind some basic commands: with “pip list” you can list all the python packages installed through pip. In the same way “conda list” lists all the python packages installed through conda. ( A good article about pip, conda and differences: link .)

Creating a Virtual Environment with pip or conda is almost the same. Let’s see both of them. (I suggest using conda which gives us more flexibility).

All the procedures illustrated here are reversible, and I describe at the end how to dismiss everything (removing the virtual environment, the kernel, etc… etc…), so you should not be concerned of testing out.

Virtual Environment With PIP

1.

Let’s create the virtual environment with pip. I like having the folder of dependencies inside my working folder, so I use the following approach: run in the terminal:

python3 −m venv myenv

Run it inside a folder of work before setting anything else.

This will create a new folder called “myenv” inside your folder of work. This contains all the dependencies of our virtual environment.

(Note: you can also use “virtualenv” in place of “venv” in the command. I tested both and they produce the same result, with virtualenv installing also Wheel library in our virtual environment myenv).

2.

Activate the virtual environment: in the terminal (placed in the working folder) run the command

source myenv/bin/activate

This will work for linux. If you are using windows, run instead

myenv\bin\activate.bat

Once it is activated, you should see from terminal

(base) nicola@nicola :~/.../folderOfWork$
source myenv/bin/activate
(myenv) (base) nicola@nicola :~/.../folderOfWork$

To deactivate the virtual environment: run the “deactivate” command

deactivate

3.

Once activated (be sure of being inside the virtual environment from command line), try to see the packages installed in our virtual environment:

pip list 

This will produce something like:

(myenv) (base) nicola@nicola:~/.../folderOfWork$  
pip list
---------- -------
Package    Version
---------- -------
pip        10.0.1 
setuptools 39.0.1

(If you used virtualenv in place of venv you have also Wheel and maybe other packages).

I suggest you to test the differences with “pip list” inside and outside the virtual environment (or in other virtual environments).

Now you can install the desired packages for your virtual environment, for example: (be sure to have the virtual environment active)

pip install numpy==1.19.0

This command will install numpy 1.19.0

4.

Now it’s time to set Jupyter Notebook to use our virtual environment.

First check which kernels you have installed for your jupyter:

jupyter kernelspec list

I have two kernels installed at the moment.

(myenv) (base) nicola@nicola:~/.../folderOfWork$ 
jupyter kernelspec list
-------
Available kernels:
  python3                                                 /home/nicola/.local/share/jupyter/kernels/python3
  python38364bitrconda633205b71ba74055bf40fd46fed5cce1
    
/home/nicola/.local/share/jupyter/kernels/python38364bitrconda633205b71ba74055bf40fd46fed5cce1

Each kernel corresponds to a Python Environment installed your machine. Not all Python Environments have a corresponding kernel. You have to make the association if you wish to use it on your Jupyter Notebook. We see immediately how to do it. First let’s note one thing.

We have two kernels as we can see from the output of the last command. When creating a new Jupyter Notebook, we can decide which Kernel to use:

As we can see we can choose among the two Kernels. A Kernel is associated with a specific Environment.

But our virtual environment has not yet any Kernel associated. To make the association, run the command: (be sure to have the virtual env active)

pip install ipykernel

(This will install the library ipykernel and all its dependencies. See what changes with “pip list”). Then:

python −m ipykernel install −−user −−name=myenv

It’s a good practice to put the name of the environment the same as the Kernel name ( “— name = myenv “, in our case).

(Important Note: the command above has to be run with the virtual environment myenv active. That command takes the actual active environment and will associate the kernel to it. If you typed the command when not in the virtual environment, don’t worry. You can revert by uninstalling and reinstalling the kernel in the proper environment. See below).

Now we made the association. Check if this indeed the case: run again the command

jupyter kernelspec list

That should produce you the list with the kernels, with the new one “myenv” added.

If everything is fine, now, by opening Jupyter Notebook and trying to create a new document, you should see:

As you can see now we also have the “myenv” kernel associated to our virtual environment.

To test that indeed we are outside of the global python, you can try to import a library that you have not installed inside myenv (e.g. inside the notebook, run “import pandas”, if you have not pandas inside your virtual env: check with pip list). This will return you error.

5.

Reverting

We installed the virtual environment and the kernel. Now let’s see how we can also remove them.

a. In order to remove the virtual environment is enough deleting the folder containing the virtual environment (“myenv” in our case).

b. In order to remove the Kernel associated to the virtual environment, run the command

jupyter kernelspec uninstall myenv

To check that is has indeed been deleted, try to list the kernels (“jupyter kernelspec list”), you won’t find anymore the “myenv” kernel.

You do not have to delete them in this order. You can independently remove one and the other.

Virtual Environment With CONDA

Conda has many advantages compared to pip. We can install a desired version of python for our virtual environment. We can also install and use pip inside conda.

1.

Create a virtual environment with conda with the following command line, inside a work folder (before setting up anything else):

conda create -n myenv python=3.8

The virtual environment is called myenv in this case. Press “y” button whenever it asks you to proceed.

Note also that we are specifying the version of python that we want for this virtual environment. You can also not specify that and it will install a default one.

Now you can check that indeed conda has created your virtual environment:

conda env list

This will list all the environments created with conda, and the paths where they are stored into.

2.

Let’s activate the virtual environment:

conda activate myenv

And to deactivate:

conda deactivate

Keep the virtual environment “myenv” activated for the next steps.

Check the installed packages through “conda list” command. We don’t have pip installed by default.

3.

You can install pip inside conda virtual environment. This makes conda very flexible.

conda install pip

Now you have pip within the virtual conda environment. Note that we can check the packages installed with conda (“conda list”) and those installed with pip (“pip list”), all inside the conda virtual environment.

3.

Now, as with the pip installation, we need to associate a kernel to our virtual environemnt. These follow the same steps as the pip installation, with few differences. The following, not always necessary, is installing ipykernel (it might come already installed. You can check with “pip list”):

pip install --user ipykernel

This will install ipykernel package. Note the difference with pip. Now we are adding “ — user” in the command.

4.

Then, exactly as in the pip case (see step 4. of the pip above), we install the kernel:

python -m ipykernel install --user --name=myenv

And check the available kernels with “jupyter kernelspec list”.

We are done, now you can open a new Jupyter Notebook and select the new kernel myenv associated to our virtual environment (see step 4. in pip for illustrations).

5.

Reverting.

If you want to remove our virtual environment “myenv” and the relative kernel:

a) In order to remove the conda virtual environment, run:

conda env remove -n myenv

You can check that is has indeed been deleted by listing the environments (conda env list).

b) In order to remove the Kernel associated to the virtual, run:

jupyter kernelspec uninstall myenv

To check that it has indeed been removed, list the kernels (jupyter kernelspec list).

End

And this is all!

Everyone in the development field had the problem of solving dependency issues at least one time in their life. Virtual Environments help us to fix this and having libraries bounded to our specifical project and not used globally for all the projects, where we might need some specific versions in order to avoid conflicts.

I remember when I had an issue with heat maps with Seaborn library, and I had to revert to a previous version and I caused many problems to the dependencies. At the time I did not know much about virtual environments and never solved the problem until a new fix to the library came… That was awkward!

It turns out that Virtual Environments are simpler than what it might seem. For who might not know them, Virtual Environments are some sort of independent boxes where we can install python (in this case) again from zero, and then decide which library to install in it (when we require some specific versions of the libraries). This comes very handy because without that you are using the global library for all your projects. Virtual Environments create, as the name suggests, a virtual box where we can operate with, indipendently by the global packages.

I hope It was helpful this article! Coming from Web Development, where with npm I had an easy time managing independent environments, I’ve found out quite some difficulties for python. I hope it will be easier for you!

Thanks for reading.

Further Suggested Reading:

Overview of python dependency management tools

Installing the IPython kernel - IPython 7.19.0 documentation

Using Jupyter Notebook extensions - Anaconda documentation