Making your C library callable from Python by wrapping it with Cython
Cython is known for its ability to increase the performance of Python code. Another useful feature of Cython is making existing C functions callable from within (seeemingly) pure Python modules.
I have recently faced at work the need to directly interact from Python with a small C library I have written. After a little research, I decided that Cython was the most fitting choice out of several existing options. While it is rather simple (which is kind of the point) to wrap C library with Cython, I have found no direct and simple tutorial for this task, so I decided to share the process.
In this tutorial we will wrap a simple C function with Cython and call it from a pure Python module, which will be agnostic to the fact its calling C code. This will require the following steps:
- Building the C library
- Installing Cython
- Creating a .pyx file in which the C function will be declared and wrapped
- Creating a setup.py file which will create a shared object that will function as an importable python module
- Building the module
- Creating a pure Python file, importing the module, and calling the wrapped function
1. Building the C Library
We will wrap the following function (files are here if you want to follow along). It is simple and not so useful, but you can literally wrap any function — which can be used to wrap lower-level system calls which are not exposed in Python!
First, we need to build a library (it may be dynamic or static, for simplicity we will build it as a static library). Create an object file and build the library:
gcc -c examples.c
ar rcs libexamples.a examples.o
Creating a makefile is always helpful for your future-self and coworkers, and will ease the build process of the Cython module:
2. Installing Cython
Simply install with pip:
pip3 install cython
3. Creating a pyx file
The pyx file is what Cython will compile to a shared object. This is the file which is written in the actual Cython language, a superset of Python, and as such mixes pure Python with C-like declarations. In the following snippet we include the declaration of the hello function, and wrap it with a Python-callable function:
4. Creating a setup.py
Cython integrates with distutils, facilitating the build of the shared object:
Notice the library, library_dir and include_dir parameters — the library name is the file name without the lib prefix and .a suffix, and the paths to the directories should obviously match the project structure. However, if all files are in the same directory, the dir parameters may be omitted.
5. Build the module
From the CLI, run the following command:
python setup.py build_ext --inplace
The shared object will be compiled, and if everything went well, a ‘build’ directory, a ‘pyexamples.c’ file and a shared object with a somewhat complex name will be created. You only need the shared object, so feel free to remove the C file and the build directory. If you want, it is interesting to take a peek at the generated C file!
Again, writing a makefile is recommended. We can use the makefile we made earlier to build everything with one simple command:
6. Calling the wrapped function
Almost done! Now we can use our C function from Python:
As you see, we import the pyexamples module as if it was regular Python module, and call the function as if it was a regular Python function — but do notice that we pass a bytes string and not a regular string, as C code essentially handles bytes strings (try using a regular string and see what happens). Now run the module and see the final result for your own,
I recommend deleting the all the build-generated files, and trying to run the pure Python module, and than using the makefile to see how it eases the build process.
You can now replace the simple hello function with more interesting functions and profit from the benefits of being able to directly call C from Python.