Gaining a better understanding of Python modules and packages

Akshar Raaj
Apr 1, 2020 · 4 min read
Image for post
Image for post

This post attempts to clear frequently occurring confusion around Python modules and packages.

You should read this post if you frequently grapple with the following errors:

  • Attempted relative import in non-package
  • No module named package.module
  • You want to understand how relative imports work

We will create two Python modules. These modules are without a package.

A Python package doesn’t come into picture unless there is an __init__.py defined.

Let’s create a directory called anydir in your home directory.

╰─$ mkdir anydir
╰─$ cd anydir

Let’s create a Python module called hello.py with following code:

def hello():
return 'hello'

Let’s create another Python module called say_hello.py with the following code:

from hello import hellodef say():
return 'saying %s' % hello()

We can start a Python shell and invoke say.

In [1]: from say_hello import sayIn [3]: print say()
saying hello

Very often developers think that since hello.py and say_hello.py are in the same directory so they could use relative import instead of an absolute import.

They change from from hello import hello to from .hello import hello.

Let’s try that. Make say_hello.py look like:

from .hello import hellodef say():
return 'saying %s' % hello()

Restart Python shell and invoke say again.

In [1]: from say_hello import say
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-1-c19beb0ca355> in <module>()
----> 1 from say_hello import say
/Users/akshar/anydir/say_hello.py in <module>()
----> 1 from .hello import hello
2
3 def say():
4 return 'saying %s' % hello()
ValueError: Attempted relative import in non-package

Shell would have raised a ValueError with message Attempted relative import in non-package.

Relative imports can only be used in a package. Since there is no package here, so it failed.

Let’s create a package called mathematics inside anydir.

╰─$ pwd
/Users/akshar/anydir
╰─$ mkdir mathematics╰─$ touch mathematics/__init__.py

We created an __init__.py inside mathematics, so mathematics becomes a package.

Let’s create a module called simple.py in package mathematics.

╰─$ vim mathematics/simple.py

Add the following content to this module.

def sum(a, b):
return a + b

Let’s create another module called advanced.py in package mathematics.

╰─$ vim mathematics/advanced.py

Add the following content to this module.

from simple import sumdef add_five_to_sum(a, b):
return 5 + sum(a, b)

Let’s start a Python shell and invoke add_five_to_sum.

In [1]: from mathematics.advanced import add_five_to_sumIn [2]: add_five_to_sum(3, 4)
Out[2]: 12

Let’s remove mathematics/__init__.py and try to invoke add_five_to_sum.

╰─$ rm mathematics/__init__.py╰─$ ipythonIn [1]: from mathematics.advanced import add_five_to_sum
---------------------------------------------------------------------------
ImportError Traceback (most recent call last)
<ipython-input-1-452b1a8afbc5> in <module>()
----> 1 from mathematics.advanced import add_five_to_sum
ImportError: No module named mathematics.advanced

Let’s add mathematics/__init__.py back. Things should work as earlier then.

╰─$ touch mathematics/__init__.py

Let’s use relative import in mathematics/advanced.py. Modify the import statement to look like the following:

from .simple import sum

Let’s invoke add_five_to_sum.

╰─$ ipythonIn [1]: from mathematics.advanced import add_five_to_sumIn [2]: add_five_to_sum(3, 4)
Out[2]: 12

Let’s try to use add_five_to_sum from inside the package i.e from the directory mathematics.

╰─$ cd mathematics╰─$ ipythonIn [1]: from advanced import add_five_to_sum
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-1-300480d69d38> in <module>()
----> 1 from advanced import add_five_to_sum
/Users/akshar/anydir/mathematics/advanced.py in <module>()
----> 1 from .simple import sum
2
3 def add_five_to_sum(a, b):
4 return 5 + sum(a, b)
ValueError: Attempted relative import in non-package

Now we are working in a module context and not in a package context. So relative imports of advanced.py raised an error.

This suggests that if you have a package then you need to work from the directory containing the package. You cannot work in the package directory itself.

Let’s come back to our base directory.

╰─$ cd ..╰─$ pwd
/Users/akshar/anydir

We can create a subpackage inside package mathematics. Let’s create a package complex inside mathematics.

╰─$ mkdir mathematics/complex╰─$ touch mathematics/complex/__init__.py

Since complex is a package, so we created an __init__.py in it.

Let’s create a module called all.py in subpackage complex.

╰─$ vim mathematics/complex/all.py

Let’s add the following content.

from mathematics.simple import sumdef add_ten_to_sum(a, b):
return 10 + sum(a, b)

Let’s invoke add_ten_to_sum.

╰─$ ipythonIn [1]: from mathematics.complex.all import add_ten_to_sumIn [2]: add_ten_to_sum(10, 5)
Out[2]: 25

We can use relative imports in mathematics/complex/all.py.

from ..simple import sumdef add_ten_to_sum(a, b):
return 10 + sum(a, b)

Restart shell and things should keep working as earlier.

All this setup is only going to work properly as long as you are working from anydir. We can consider anydir as our base directory.

If you try to work from directory mathematics things will break. It’s important to give a proper thought to your base directory and define packages in it and do imports accordingly.

Feel free to ask any question on modules and packages and I would be glad to answer.

The Startup

Medium's largest active publication, followed by +752K people. Follow to join our community.

Akshar Raaj

Written by

Engineer | Open Source | Blogger | Speaker. My posts have been read over a million times. https://www.agiliq.com/authors/#Akshar https://bit.ly/2TaRqji

The Startup

Medium's largest active publication, followed by +752K people. Follow to join our community.

Akshar Raaj

Written by

Engineer | Open Source | Blogger | Speaker. My posts have been read over a million times. https://www.agiliq.com/authors/#Akshar https://bit.ly/2TaRqji

The Startup

Medium's largest active publication, followed by +752K people. Follow to join our community.

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store