Numba — Making Numpy 50x Faster
Python is slow, as it is interpreted at run-time and not compiled to native code. People have tried to get a compiler for Python for quite sometime now but Python being a dynamic langauage, most of the efforts have been far from successful — or so I thought.
Enter Numba!
Concept: If we cannot compile the entire python code, compile parts of it without many changes your existing code.
The purview of this article is restricted to JIT but you should definitely check out other options that Numba provides. Using JIT as a decorator while defining your functions is the easiest way of integration with Numba.
We can also enable eager implementation by defining the input and output dtypes. Further, Numba can call other compiled functions while compiling a function:
Now lets look at the compilation options that JIT decorator of Numba provides:
- nopython: When set to True, produces a faster code and upon failure falls back to python interpreter
- nogil: Removes the python global interpreter lock (GIL) as compilation happens in native code and not python objects.
- cache: Writes the compiled cache to a local file so that on code rerun, the compiled function is used.
- parallel: Use of multiple cores on the machine
You can read them in detail and see their implementation here.
Another version of the jit decorator is njit which is equivalent to @jit(nopython=True). A few other performance improvement tips using jit can be found here.
Now let us look at how Numba performs:
I generate two numpy arrays with random values and define the function to multiple values of both arrays, one without jit and one with:
Upon timing the withOutJIT function over 7 runs, the time it took was over 4.6 ms:
The same function with JIT compiler enabled took 87.4 micro seconds, 52x faster than earlier:
As the data size increases and computation becomes more challenging, Numba would make your code run faster than pure Python, without making any changes to your code. Get started with Numba today!