Accelerate Your scikit-learn Applications
Faster Experimentation with Predictable Behavior
Oleksandr Pavlyk (Intel Corporation) and Olivier Grisel (INRIA)
Intel oneAPI AI Analytics Toolkit (AI Kit) includes Intel® Extension for Scikit-learn that accelerates a selection of common scikit-learn estimators (e.g., logistic regression, singular value decomposition, principal component analysis). These functions are built by using Intel oneAPI Data Analytics Library (oneDAL) so they achieve performance close to that of equivalent C++ programs.
oneDAL’s (previously known as DAAL, or Data Analytics Acceleration Library) performance comes from efficient use of multiple CPU cores, cache-friendly blocking, and effective use of processor instruction sets. It is tuned to run best on Intel processors. Improved scikit-learn performance benefits users in shortened model development iteration cycles and reduced cost of training. Improved software engineering in the library also results in a smaller memory footprint, which allows users to tackle larger machine learning problems with their existing hardware.
Figures 1 and 2 show speedups of the accelerated scikit-learn with Intel® Extension for Scikit-learn over the base library, as measured with scikit-learn_bench. Figure 1 compares the multithreaded, accelerated scikit-learn against the best performance of the base scikit-learn between n_jobs=1 and n_jobs=-1. The scikit-learn user needs to know the algorithm details to use the n_jobs setting to improve performance for some functions (e.g., using a non-default value of n_jobs for LogisticRegression is detrimental to performance). Scikit-learn developers are working to improve the user experience.
Running the accelerated scikit-learn sequentially shows that many algorithms in the base scikit-learn have room for performance improvement, notably training and inference of SVC as well as training of LinearRegression (Figure 2).
The pursuit of performance can sometimes sacrifice correctness if the developer isn’t careful. To insure that the accelerated scikit-learn lives up to the high standards of the scikit-learn user community, the accelerated version is being required to pass the scikit-learn test suite. A system to run these tests was developed in collaboration with scikit-learn core developers, Olivier Grisel and Jérémie du Boisberranger. Testing is done with the currently released scikit-learn and the current master sources.
The testing is performed in CircleCI so an interested user has easy access to the testing logs for further inspection.
Special attention is paid to insuring deterministic, mathematical equivalence between the accelerated scikit-learn and the base version. Mathematical equivalence means solutions obtained by both versions agree within the tolerance specification of the solver. Such a cross checking has resulted in a feedback to improve scikit-learn’s own test suite, e.g.: scikit-learn/#12738, #12263, and #13992.
This collaboration also given scikit-learn developers a better understanding of the performance of their implementation. For instance, @jeremiedbb completely refactored the k-means implementation using Cython to improve multithreaded scalability (scikit-learn#11950). This work is now part of the 0.23 release of scikit-learn.
You can accelerate your own scikit-learn installation by installing Intel® Extension for Scikit-learn package separately or as part of the AI Kit, which includes stock scikit-learn with all the latest optimizations and is distributed through many common channels, including Intel’s website, YUM, APT, Anaconda, and more. Select and download the distribution package that you prefer and follow the Get Started Guide for post-installation instructions.
You can also use the conda package manager or pip to install only Intel® Extension for Scikit-learn package using:
conda install scikit-learn-intelex -c conda-forgepip install scikit-learn-intelex
Once Intel® Extension for Scikit-learn is installed, you can accelerate your scikit-learn installation (version >=0.19) in either of two ways:
python -m sklearnex your_application.pywhich is great for running tests, and for quick experimentation. You can also do so explicitly in your script:
from sklearnex import patch_sklearn
patch_sklearn()from sklearn.svm import SVC # your usual code without any changes
…
Patching is accompanied by informational message:
In [1]: from sklearnex import patch_sklearnIn [2]: patch_sklearn()
Intel(R) Extension for Scikit-learn* enabled
(https://github.com/intel/scikit-learn-intelex)
Follow us on Medium and GitHub to keep up with the latest updates.
Try accelerating your Scikit-learn code with Intel® Extension for Scikit-learn and feel free to submit your feedback on GitHub.
