Python import module from root directory?

Photo by Paul Teysen on Unsplash

Introduction

Import module in Python is fairly easy, here is example.

import my_module as module
from my_module import say_hello

However it will throw an error in certain case.

Recently, I encounter this error during development. In a project I have few services that run in docker, which is a containerized light weight virtual machine. Those services may have common functionalities therefore I make them as a Python module but the example above for importing Python module is no longer work.

The problem

python_modules_example/
┣ app/
┃ ┣ app_modules/
┃ ┃ ┣ utils.py
┃ ┃ ┗ __init__.py
┃ ┗ app.py
┣ common_modules/
┃ ┣ utils.py
┃ ┗ __init__.py
┣ .gitignore
┗ main.py

The __init__.py files are required to make Python treat directories containing the file as packages. See here.

A project file and folder structure above. There is no problem to import modules from common_moduls and app_modules within main.py.

main.py

import common_modules.utils as common_utils
import app.app_modules.utils as app_utils

if __name__ == "__main__":
    common_utils.say_hello()
    app_utils.say_hello()

But this is no longer the case for app.py.

app.py

# import module from outside of app folder
import common_modules.utils as common_utils

# import module from inside app folder
import app_modules.utils as app_utils

if __name__ == "__main__":
    common_utils.say_hello()
    app_utils.say_hello()

If I run app.py then an error occur.

Traceback (most recent call last):
  File "./app/app.py", line 2, in <module>
    import common_modules.utils as common_utils
ModuleNotFoundError: No module named 'common_modules'

The error says common_modules not found. Take a look at directory path of common_modules, it is under root directory.

Why

The reason it throw an error says module not found is because common_modules is not at same or in subdirectory of app.py.

When we run app.py, Python will search for modules at same directory or in subdirectory of app.py and any directory other than that will not be included. That explained why main.py work fine but not work for app.py.

If we comment out all code in app.py and add these two lines.

import sys
print(sys.path)

It will print a list of module search paths as string in my local machine.

[
'C:\\Users\\tomne\\Desktop\\python_modules_example\\app', 
'C:\\Users\\tomne\\AppData\\Local\\Programs\\Python\\Python38\\python38.zip', 
'C:\\Users\\tomne\\AppData\\Local\\Programs\\Python\\Python38\\DLLs', 
'C:\\Users\\tomne\\AppData\\Local\\Programs\\Python\\Python38\\lib', 
'C:\\Users\\tomne\\AppData\\Local\\Programs\\Python\\Python38', 
'C:\\Users\\tomne\\AppData\\Local\\Programs\\Python\\Python38\\lib\\site-packages'
]

A module search path is initialized when Python starts. This module search path may be accessed at sys.path.

The PYTHONPATH environment variable is often used to add directories to the search path. If this environment variable is found then the contents are added to the module search path.

see here.

As you can see app directory is at first in list but there is no sight of common_modules directory.

Solution

The solution is simple we can add root directory to the list so Python will search entire directory.

app.py

import os
import sys
# insert root directory into python module search path
sys.path.insert(1, os.getcwd())

# import module from outside of app folder
import common_modules.utils as common_utils

# import module from inside app folder
import app_modules.utils as app_utils

if __name__ == "__main__":
    common_utils.say_hello()
    app_utils.say_hello()

We insert the root directory into the list with one line of code before we importing any modules.

sys.path.insert(1, os.getcwd())

os.getcwd() return a path to current working directory in string

Now when we run app.py then it work as we expected without issue.

Here is the example project.

Conclusion

Often we do not need to do this. However in some cases we might need to import a module from different directory than where we run our Python script. And this solution solve the problem we encounter.