Take a “pip” into my programming toolbox

Dependency management in Python will have any programmer spoilt for choice and in this post I will be talking about some awesome tools that I use to get and stay productive.

1. virtualenv

Virtualenv is a tool used to create isolated Python installations. This means you can have Python 2.7 and Python 3.6 installed on a single system without any issues. Its very simple to get started with it:

virtualenv [name of virtual environment] 
virtualenv venv - this creates a virtual environment named "venv"
source venv/bin/activate - this activates the virtual environment
and in an active environment you can now run pip install "package" to install any python package in that active environment.
deactivate - this deactivates the virtual environment

2. pipreqs

A “requirements file” is a file within a Python projects that contains a list of installed packages within a virtual environment. Its role is to communicate application dependencies to:

1. Machines (build and deployment environments)
2. People (developers, operations teams)

Normally with pip you can generate a requirements.txt file with this:

pip freeze > requirements.txt

on inspecting the file you will notice that the output isnt human friendly i.e. it contains some not so important information that a programmer might not need to know about. A sample output of the above command might look like this:

appdirs==1.4.3
asn1crypto==0.22.0
attrs==17.2.0
Automat==0.6.0
certifi==2017.4.17
cffi==1.10.0
chardet==3.0.3
click==6.7
constantly==15.1.0
cryptography==1.8.2
cssselect==1.0.1
docopt==0.6.2
first==2.0.1
graphene==1.4
graphql-core==1.1
graphql-relay==0.4.5
hashin==0.9.0
idna==2.5
incremental==17.5.0
jedi==0.10.2
leak==1.0.3
lxml==3.7.3
packaging==16.8
parsel==1.2.0
pip-tools==1.9.0
pipdeptree==0.10.1
pipreqs==0.4.7
pipsi==0.9
promise==2.0.2
prompt-toolkit==1.0.14
ptpython==0.39
pyasn1==0.2.3
pyasn1-modules==0.0.8
pycparser==2.17
PyDispatcher==2.0.5
Pygments==2.2.0
pyOpenSSL==17.0.0
pyparsing==2.2.0
queuelib==1.4.2
requests==2.17.3
Scrapy==1.4.0
service-identity==17.0.0
six==1.10.0
termcolor==1.1.0
Twisted==17.1.0
typing==3.6.1
urllib3==1.21.1
virtualenv==15.1.0
w3lib==1.17.0
wcwidth==0.1.7
yarg==0.1.9
zope.interface==4.4.1

Hence pipreqs. It is a tool that generates a “requirements.txt” file based on explicit imports within a Python application. Running this command:

pipreqs .

yields this

Scrapy==1.4.0
Sanic==0.1.9
Flask==0.12.2
SQLAlchemy==1.1.10pip-compile

These are packages explicitly imported in my code.

3. pip-compile

This is a better “pip freeze”. It takes a requirements.in file (for people) and generates a frozen requirements.txt file (for machines). So you can do:

pipreqs . --savepath requirements.in
pip-compile

To update all outdated modules using pip-compile:

pip-compile --output-file requirements.txt requirements.in req

4. pip list — outdated

This is used to list all installed packages that are not up to date with the version on the PyPI index.

5. pip-compile — generate-hashes

This is used to ensure that you actually install all the packages that you wanted to, especially in the face of truncated downloads or network issues.

6. pipdeptree

This tool is used to display a dependency tree like structure of all the installed packages in a virtual environment.

pipdeptree --reverse 

is used to display all modules with the reasons why they are installed i.e. dependencies

7. leak

This is used to display meta information of packages on the PyPI index.

leak sanic
================================================================================
Sanic
A microframework based on uvloop, httptools, and learnings of flask
--------------------------------------------------------------------------------
Author:              Channel Cat
Author mail:         channelcat@gmail.com
Available versions:  18
Home page:           http://github.com/channelcat/sanic/
================================================================================
(0.5.4) Most recent. 09/05/2017 00:49 release date
(0.5.2)
(0.5.1)
(0.5.0)
(0.4.1)
(0.4.0)
(0.3.1)
(0.3.0)
(0.2.0)
(0.1.9) Most popular. 1053 downloads
(0.1.8)
(0.1.7)
(0.1.6)
(0.1.5)
(0.1.4)
(0.1.3)
(0.1.1)
(0.1.0)

For further reading check out Pipfiles and Pipenv which is built on Pipfiles and virtualenv.