TL; DR: v8.py
PyMiniRacer brings a compiled V8 interpreter and a simple interface to the Python community:
Our needs
Sqreen needs a stable, performant and embeddable JavaScript solution for Python. We use it to run security logic in the Sqreen agent that our customers install in their applications. This has several advantages. In a first hand, execution is confined in the JavaScript VM, which enforces reliability. On a second hand, the security logic we build can be shared across platforms. This allows Sqreen to transparently monitor and enforce security in our customer’s applications, whatever the platform is.
The Ruby ecosystem already provides similar solutions. As a matter of fact, the generic JS backend wrapper ExecJS is even part of Ruby on Rails core.
Since we have found no project with such stability nor popularity in Python, we decided to write our own.
Embedding V8
Basically, we need three things:
- A JS engine;
- A binding between the JS engine and Python;
- The Python interface.
First, you need a JS engine. Choosing V8, the Google JavaScript engine, initially built for Chrome, is a no brainer. It is used in both Chrome and NodeJS. These projects have emphasized the speed, quality, and stability of the project.
Then you need to interface V8 with Python. In Python, there are two ways to do it. We could either use the Python C API, or ctypes. We have chosen ctypes since it allows a generic and more portable implementation. Indeed, no Python header is needed for compilation. Furthermore, it reduces the number of targets that need maintenance for the binary distribution.
As a reference implementation, we chose the mini_racer gem, written by Sam Saffron. It is a simple one, with everything we needed, that allows to:
- Create isolated contexts;
- Eval code on it that may define functions.
Finally, the Python library. That’s the part that will be shown to the package users. We designed the simplest API possible, that allows to create a context, and either eval or call functions on it.
Introducing: PyMiniRacer
Check out the Github page of PyMiniRacer.
PyMiniRacer is open source and released under the ISC License.
Usage
Installation is straightforward on OSX and most Linux distributions:
$ pip install py_mini_racerThe interface is very straightforward:
Binary distribution
Building V8 is a long process. You need a system with a C++ compiler. Then, to download a few gigabytes of V8 source code from Google repository. Eventually, you compile hundreds of C++ files and link them together.
PyMiniRacer does all of that for you :). Thanks to recent efforts in Python binary packaging, we have been able to package PyMiniRacer as Python wheels for the most common platforms and Python versions. You only need to pip install it, and you are ready to go!
If your version is not supported yet, please create a GitHub issue.
We have packaged a 4 MB binary version of PyMiniRacer which installs instantly. These should help you embed V8 into your Python application in a very easy way.
Performance and reliability
PyMiniRacer bundles a plain V8. It uses its C++ interface to communicate with it. Executing a JavaScript string in a given context is done with the following steps:
- Take the JavaScript input as a Python string (in the MiniRacer.eval Python method);
- Pass it to V8 after a String::NewFromUtf8 conversion;
- Then V8: compiles it with Script::Compile; & executes it with Script::Run;
- PyMiniRacer converts the result to an abstract binary interface;
- Return it to Python that will convert it using ctypes.
There is virtually no overhead since in step 1. we only copy a string, and in step 4. we perform a very light and low-level conversion from ctypes to Python base objects (strings, numbers, dates, arrays, and dicts).
Regarding reliability, there are very few places where a failure can occur, since most of the work is performed by V8.
About the author
Jean-Baptiste Aviat spent half a decade hunting vulnerabilities at Apple, helping developers solve them, and developing security software. He is now CTO at Sqreen.
Originally published at blog.sqreen.io.
