What Is the Difference Between a Module, a Package, a Library, and a Dependency?

When I first began programming, I thought you had to do everything by scratch.

Do you want to scrap some text off a website? You’ve gotta build it. Do you want to create a table? You’ve gotta build it. Do you want to do machine learning? Well, I guess you have to learn how to build that too.

But then I learned of the wonderful world of open-source programming, and the beauty of the word import. However, there was a whole slew of equally wonderful jargon I was met with; modules, packages, libraries, and dependencies.

I hope to disambiguate these terms, which for the past too long I have been using interchangeably.

The Module: The basic unit of importation

Imagine you wrote some helpful functions, for convenience let’s say in Python. One takes in a list of words and gives back a list of only the ones that have the letter “a” in them. The other takes in a list of words and returns only the ones that have an even number of characters. (All of the code in this post is on my GitHub)

def has_a(word_list): # define a function that takes in a list 
    a_list = [] # make an empty list
    for word in word_list:      # for each word in the inputted list
        if “a” in word.lower(): # if it has the letter a in it
            a_list.append(word) # add it to the a_list
    return a_list
def is_even(word_list): # define a function that takes in a list
    even_list = [] # make an empty list
    for word in word_list:         # for each word in the list
        if len(word) % 2 == 0:     # if it has an even length
            even_list.append(word) # add it to the even_list
   return even_list

Within the same file, you’d be able to call upon these functions to do all of your “a”-finding and even length needs.

However, if you were writing another script that needed these functions, you wouldn’t be able to access them or another other feature from outside the original file. You would have to copy and paste everything into the new script.

If you only have one or two functions that isn’t too bad of a process. Now imagine you had a collection of a hundred classes and functions. That would get tedious quickly.

Modern languages allow for you to import functionality from one file to another.

So what if I wanted to use these functions in another file? I could first save them in their own file, let’s call it helpful.py. Then I could call on it’s wonderfulness with the import command. I’d be able to do all of the “a”-finding in another file without having to redefine the function.

import helpful
my_list = [“Small”, “Medium”, “Large”, “Python”]
# Only grab the elements with “a
my_a_list = helpful.has_a(my_list)
# Which should be Small and Large 
assert my_a_list == [“Small”, “Large”] 
 
# Only grab the elements of even length
my_even_list = helpful.is_even(my_list) 
# Which should be Medium and Python 
assert my_even_list == [“Medium”, “Python”]

These single files worth of similar imported functionality are modules. Python has a large collection of modules that are distributed along with Python known as the Python Standard Library

Notice how I had to specify the file name helpful before I used its functions. If I wanted to instead grab specific functions from helpful.py, I could use the from command.

from helpful import has_a
my_list = [“Small”, “Medium”, “Large”, “Python”]
my_a_list = has_a(my_list)
assert my_a_list == [“Small”, “Large”]

Additionally, you can grab every name from a file by using a *.

from helpful import *

However, this is generally frowned upon because it makes your code harder to understand. Additionally, an unexpected name from the external file could conflict with one of the names in your file.

Finally, you can use the as keyword to define a name you want to use for a module or feature of a module.

import helpful as hlp
my_list = [“Small”, “Medium”, “Large”, “Python”]
my_a_list = hlp.has_a(my_list)
assert my_a_list == [“Small”, “Large”]

The Package: A module of modules

Of course, just like how we could extend the functionality of our script with a module, you could imagine that we may need a whole suite of related modules to accomplish some complex functionality.

A package is a special type of module that contains many other modules in a way that makes it straightforward to import from. This shields each module from having to worry about the naming schemes of other modules conflicting with theirs.

My_package/
    — __init__.py
    — module1.py 
    — module2.py 
    — module3.py

The __init__.py file is used to mark directories or folders on your computer as Python package directories (it is actually no longer necessary as of Python 3.3). It can be empty, or it can include some useful functionality such as running initialization code, or define which modules get imported with * as we will see later.

Now imagine we wanted to create a package that was all about images. We may need some modules that are related to the actual file formats those images could come in (png, jpg, svg). We may need other modules that are related to filters we could add to those images (black and white, sepia, dog filter).

To keep these organized, we could create subpackages within our main image package that further subdivide it.

image/
    — __init__.py
    formats/
        — __init__.py
        — read_png.py
        — read_jpg.py
        — read_svg.py
        …
    filters/
        — __init__.py
        — sepia.py
        — black_and_white.py
        — dog.py
        …
…

To import these modules, you use a “dotted module name” convention. Imagine the module dog.py has a function called apply_tongue.

import image.filters.dog
dog_img = image.filters.dog.apply_tongue(my_img)

Notice how I had to invoke it with its full name. Alternatively, you can also import specific modules.

from image.filters import dog
dog_img = dog.apply_filter(my_img)

Or you could import the specific feature you need from a module.

from image.filters.dog import apply_tongue
dog_img = apply_tongue(my_img)

__all__ is a list of module names that you can put in the __init__.py to determine which modules gets imported with from <module> import *.

Let’s say for the __init__.py of the filters subpackage we define:

__all__ = [“sepia”, “dog”]

Then, whenever we import all we only import the sepia and dog filter modules:

# import all from filters according to its __init__.py
from image.filters import * 
assert sepia # The sepia filter can be used
assert dog # and so can the dog filter
try:
    assert black_and_white
except NameError:
    print(“black_and_white isn’t imported!”) # but not a b&w one

Python is home to a large and growing collection of packages that can be found (along with libraries and simple modules) in the Python Package Index.

The Library: A collection of imported functionalitwidth: 100%

Some Machine Learning Libraries

Unlike a module or a package, a library does not have as precise a definition, at least in Python. It is a generic term that is used to describe a bundle of code that can be used by many different applications. It could be made up of a single module or several packages.

In Python, it is not uncommon for libraries to refer to a collection of modules and packages which has been “published” or made available for use in an open-source way. Libraries can range in function from getting information from websites (requests) to machine learning (TensorFlow/scikit-learn).

The Dependency: A needed functionality

Finally, a dependency is simply something, whether it’s a module, package, or library, which an application needs to run.

For example, If I am creating a machine learning model, I may need NumPy to create efficient arrays, TensorFlow and Keras to create a model, Pandas to store information about our model and its trials in a Pandas DataFrame, and Matplotlib to visualize that information.

A Pandas DataFrame

All of these dependencies are pieces of code written by other developers that have been put online for other people to use. Without them, it would take an immense amount of time to even begin trying a project such as a machine learning model.

Imagine having to create your own way of storing data, your own calculus to handle machine learning calculations, and your own way of graphing results. That would suck.

All hail GitHub. \o/

Conclusion

The open-source culture of programming is an amazing thing. Large companies like Google make their cutting edge libraries available to everyone in the hope that people will build upon it in new and unique ways.

So go out there and make some new and amazing programs. Build upon the shoulders of all the libraries, packages, and modules that have been built before you! Next time you need to do some complicated task to run your code, maybe check GitHub first.

Thanks for reading!

• Jordan

Originally published at https://www.jordantwells.com.