A Simple Way to Build Command Line Interface Applications in Python

In this post, I will explain how to build a simple CLI application using python. Prior programming experience in any language and familiarity with computers will be helpful to understand this post.

I was recently asked to automate a number of tasks for the quality assurance team at work. Automating processes is essential for teams that want to reduce the errors associated with performing tasks manually. The main requirement was that it was simple to use. I felt it was also important to make the application extensible as I might be adding new functionalities to it in the future. With those requirements in mind, I built a command line interface (CLI) application with multiple entry points and tests.

What are we building.

A simple tool that converts numeric numbers to their word form. E.g, given 129, the tool will display one hundred and twenty nine. The tool will also offer the option to display a help message, split numbers into groups before converting and verbose mode. To learn more about verbose mode see this post.

To display help messages we execute the command convert -h . -h is the option/flag that tells the application to display the help message to standard output.

(venv) Ibukuns-MBP:cliApp ioluwayo$ convert -h
usage: convert [-h] [-g G] [-v] number
This script converts numbers to words.
positional arguments:
  number         The number to convert to words.
optional arguments:
  -h, --help     show this help message and exit
  -g G           Split the number int g groups before converting.
  -v, --verbose  Verbose logging.

The help message contains all the information required to use the application properly. Optional arguments are not required, but positional arguments are.

usage: convert [-h] [-g G] [-v] number
[-g G] means a value is expected along with this option.
number is the only positional argument.

To convert 100 to one hundred you execute the command convert 100

(venv) Ibukuns-MBP:cliApp ioluwayo$ convert 100
one hundred

In verbose mode.

(venv) Ibukuns-MBP:cliApp ioluwayo$ convert 100 -v
convert.__main__: INFO     About to convert 100 , to words.
one hundred
(venv) Ibukuns-MBP:cliApp ioluwayo$

As you’d expect, the application provides information on what it is doing.

In Group mode.

(venv) Ibukuns-MBP:cliApp ioluwayo$ convert 100 -g 2
ten, zero
(venv) Ibukuns-MBP:cliApp ioluwayo$

The application splits the number into 2 groups. 10 and 0.

Executing the command convert 100 -g will result in an error as the -g option requires an argument.

With both Verbose and Group mode.

(venv) Ibukuns-MBP:cliApp ioluwayo$ convert 100 -g 3 -v
convert.__main__: INFO     About to convert 100 , to words.
one zero zero
(venv) Ibukuns-MBP:cliApp ioluwayo$

Alright, so that is it for the functionality. In addition to that we would also look at how to write end-to-end tests for CLI applications. Without tests, attempts to extend software with new functionality becomes risky. You are unable to determine if the new functionality has compromised old functionalities.

Project Structure.

While there are many ways to structure python projects, this is my preferred structure for CLI applications.

├── app2 # independent application
│   ├── __init__.py
│   └── __main__.py
├── convert # another independent application
│   ├── __init__.py
│   └── __main__.py
├── requirements.txt # contains all required python libraries
├── setup.py # Entry points are defined here for each application.
├── tests
│   ├── __init__.py
│   ├── app2-tests.py
│   ├── convert-tests.py
│   └── utils-tests.py
└── utils
    ├── __init__.py
    └── moduleLogger.py

We need the following libraries. I am using python 2.7, Python 3 will also work.

inflect==0.2.5
nose==1.3.7
six==1.11.0

Install them using pip. I recommend using a virtual environment. To learn about virtual environments see this article.

setup.py

This is a very important module and it is where a lot of configurations and metadata are set. It is required for installing and packaging python applications using pip and setuptools. See this article for more information.

In setup.py Lines 10 and 11 are very important. They tell python to execute the main() method in convert.__main__ and app2.__main__ when the convert and app2 commands are typed into the console respectively. You can see convert and app2 as aliases for the methods to be called.

With the configurations in setup.py our application can be installed now.
We will install the application in editable mode while we are still working on it. When you install an application in editable mode, you do not have to reinstall it whenever you make changes to the code. Python executes the newest version of your code. Pretty cool.

Execute the command pip install -e . from the root directory of the project. Do not omit the . as it tells pip to look in the current working directory for setup.py which contains the information necessary to install the package.

__init__.py

This module (usually empty )indicates that a directory is a python package. As from python 3.3, it is no longer required, as python treats all directories as packages.

App2/

This package is just there to demonstrate multiple entry points for command line apps. There is no code in it. It demonstrates how easily the project could be extended with new functionality.

Convert/

This is the package for the convert application that we will build.

convert/__main__.py

The cli() method handles the command line interface. The argparse library is used to specify the options and arguments for the application. Since logging functionality will be required by more than one application, the util/moduleLogger.py contains a method for initializing loggers.

Utils/

This package contains modules that are required by other applications. Once a functionality is required by more than one application, It is a good idea to put it here in order to avoid writing duplicate code.

utils/moduleLogger.py

See this article to learn more about logging levels and good practices.

tests/

This package contains tests for each application. Tests make it easy to extend and maintain software. For more on why testing is important see this post.

The convert_one_hundred() method demonstrates how command line applications can be tested. The subprocess module allows you to spawn new processes, connect to their input/output/error pipes, and get their return codes. See its documentation here.

To run the tests, we will use nose. In the root directory, execute the command nosetests --exe -v. The --exe option tells nose to look for tests in python modules that are executable. The -v option is for verbose logging just like in our convert application. See the nose documentation for more on testing with nose.

requirements.txt

This file contains a list of libraries that are needed for the applications in the project to run successfully. See this article to learn more about pip.

Execute the command pip freeze > requirements.txt to copy all required libraries and their versions into the file. This way it is easy to install all the libraries the application requires in a new machine or environment just by executing the command pip install -r requirements.txt

Packaging

After building something awesome, it’d be nice to share it’s awesomeness with others. Python makes this easy through setup.py.

You will notice that lines 8 and 9 are new. install_requires and python_requires specify the minimum dependencies and the version of python required for the project to successfully run respectively. You might wonder what the difference between requirements.txt and install_requires is. See this article for more on that. In summary, install_requires specifies the ‘minimum’ dependencies for a project while requirements.txt specifies detailed information about replicating the python environment for the project which is necessary for development work.

To package the project for distribution we will use a python library called wheel. Wheels are great for fast installations as they are already built. You can learn more about wheels here. Execute the following commands.

pip install wheel 
python setup.py bdist_wheel --universal 

Look in the root directory of the project and you’d see a new directory called dist. It contains a file that looks like CliApp-0.10-py2.py3-none-any.whl. This is your packaged python application that can be distributed. The snippet below shows a sample installation and use.

pip install CliApp-0.10-py2.py3-none-any.whl
Processing ./CliApp-0.10-py2.py3-none-any.whl
Collecting inflect (from CliApp==0.10)
  Using cached inflect-0.2.5-py2.py3-none-any.whl
Installing collected packages: inflect, CliApp
Successfully installed CliApp-0.10 inflect-0.2.5
Ibukuns-MBP:dist ioluwayo$ convert 100
convert.__main__: INFO     About to convert 100 , to words.
one hundred

That’s it, tools like pip, argparse and wheel make it trivial to build CLI applications in python. All the code in this post can be found on github.