PowerShell — Creating Virtual Environments with Different Python Versions for Your Projects
1. INTRODUCTION
When I first started learning Python, I went through the motion of clicking all the next buttons to install the latest and greatest version to my computer without worrying where it was installed. Similarly, whenever a library was needed, I just conveniently execute the be-all-end-all pip install command and trusted that everything would be settled. Yes, my codes could run fine and so long as I can complete whatever task at hand, it was enough.
However, when I had a chance to be exposed to the proper software engineering work environment, I could not be more wrong. Concepts about version control, virtual environment, and compatibility came into the picture. I could not get many things to be installed and run on my local machine due to lack of required libraries, or the specified library version was incompatible with the Python version in my computer.
In this article, I attempt to outline some steps I found useful to set up different virtual environments, with different Python versions and compatible library versions. Getting these out of the way first-thing whenever you start on a new project will ensure smoother transition from/to other colleagues. Also, good house-keeping is a habit that will make our lives easier in the future.
2. WHAT IS A VIRTUAL ENVIRONMENT
To put it simply, a virtual environment is a stand-alone, self-sufficient developing environment that contains the Python interpreter and necessary packages/libraries/dependencies that a project needs to run. When a script of this project is executed, it imports the relevant libraries from the same virtual environment. Another project will not share anything with this project and will use its own virtual environment Python interpreter and dependencies. By this segregation, we know and can control precisely which versions are being used in each environment, and for which project. This will help minimize incompatibility issues.
For more details, check out the article below
3. SETTING UP SOME LOGISTICS
Now, imagine the problem statement is that we have 2 projects that will have to be developed on 2 different Python versions — 3.7 and 3.8. They will have different supported library versions. Thus, our aim is to create 2 virtual environments and install the respective requirements.
For this demo, I installed 2 versions of Python to a specific parent folder on my local drive below. You can install them anywhere you want, just remember their locations for later use. For beginners, when launching the installer, you can stick with the suggested location under Install Now or customize to your own preferences.
In addition, I will be using Windows Power Shell (in VSCode) for convenience. You can easily find equivalent commands for Mac/Linux online.
Imagine we have 2 project folders in our machine as listed below. We will create a virtual environment with Python 3.7 for the test_project_37 , and Python 3.8 for the other project. They have two different requirements.txt files as well. We put these two text files in the respective project folders.
Required versions for test_project_37 :
numpy==1.21.5
pandas==1.3.5
opencv-python==4.5.2.52Required versions for test_project_38 :
numpy==1.23.0
pandas==1.5.2
opencv-python==4.5.2.52The starting location of the PowerShell terminal, after directing VSCode to the correct location is:
PS C:\Users\phuon\ML Projects>4. CREATING VIRTUAL ENVIRONMENTS
Now let’s move to the first project folder test_project_37and execute the following command. What it does is going to the location where we installed the Python 3.7 in section 2 above and looks for the Python interpreter (python.exe), “copies” it and creates a virtual environment folder called venv37 via the -m venv syntax. This is an important step before installing any libraries into our environment, as it boils down to which library versions the desired Python version supports.
PS C:\Users\phuon\ML Projects\test_project_37> C:\"Program Files"\Python\Python37\python.exe -m venv venv37After this, you will see the new folder venv37 created in this project folder. There, once the virtual environment is activated, Python will store all future libraries. Next, we will activate this environment by calling the activate.ps1 file inside the Scripts folder.
PS C:\Users\phuon\ML Projects\test_project_37> venv37\Scripts\activate.ps1If this is done correctly, you will see the project folder address being suffixed with the virtual environment name in brackets:
We can check the version of Python by the following command. The version 3.7 is rightly returned.
(venv37) PS C:\Users\phuon\ML Projects\test_project_37> python -V
Python 3.7.9With this, we can go ahead and install the libraries specified for this project by the following command. These libraries will be installed into the venv37 folder.
(venv37) PS C:\Users\phuon\ML Projects\test_project_37> pip install -r requirements.txtNow we are done with the first environment, we can proceed to do exactly the same steps for the other environment with Python 3.8, with only difference being the location of the Python interpreter. Of course before that, you will need to deactivate the current virtual environment, and cd to the other project location.
5. HOW WILL THE NORMAL `PIP INSTALL` BEHAVE?
Once we are in a virtual environment, whenever we use pip install to install any library, if we don’t specify the desired version of the library, pip will automatically install the latest version that is compatible with the python version inside the virtual environment. Let’s see this by trying installing scikit-learn into our 2 environments above.
You can see that scikit-learn-1.0.2 was installed for Python 3.7 while that of Python 3.8 is scikit-lean-1.2.0 — which is also the latest version at the point of writing. Taking a quick look under the Meta section at the pypi.org website for this library, we see that it requires Python ≥3.8.
Navigating to the release history and doing a quick check, we can easily find that from 1.1.0, scikit-learn only supports python >=3.8, hence the latest version that will work for python 3.7 is the predecessor, which is 1.0.2. this is exactly what we see above as 1.0.2 was installed for our python 3.7 .
Of course you can always install the specific version of the package by adding ==<version number> after the pip install <package> command above.
6. FREEZING THE DEPENDENCY VERSIONS
Lastly, for completion sake, imagine we have done all the work in our local environment and want to create the requirements.txt file for others to use and install on their virtual environment, so that there is no compatibility issue. Besides telling them the python version in use (e.g., 3.8) we need to “freeze” the current package versions and export them to a text file. let’s call it “requirements38.txt” for easy differentiation with the previous file. Now the updated text file is produced in our project folder.
(venv38) PS C:\Users\phuon\ML Projects\test_project_38> python -m pip freeze > requirements38.txt7. CONCLUSION
I hope this article will help clear some doubts regarding why we should use virtual environments, how to set them up for specific Python versions, and how to install requirement libraries. Have fun and any feedback is welcomed.
Disclaimer: All opinions and interpretations are that of the writer, and not of MITB. I declare that I have full rights to use the contents published here, and nothing is plagiarized. I declare that this article is written by me and not with any generative AI tool such as ChatGPT. I declare that no data privacy policy is breached, and that any data associated with the contents here are obtained legitimately to the best of my knowledge. I agree not to make any changes without first seeking the editors’ approval. Any violations may lead to this article being retracted from the publication.
