Contribute to the Open Neural Network eXchange (ONNX)
The ONNX community is the place to be for any developer interested in ML.
I attended the ONNX community meeting on April 9, 2020, and there it became apparent to me that this is a great community on the forefront of modern open source machine learning (ML). I want to tell you about ONNX, what makes it special, and share with you things I learned at the community meeting in regard to ongoing work on the standard and ways for open source contributors (like you!) to join the effort. Lastly, I’ll talk about how my experience with Predictive Model Markup Language (PMML) contributes toward the progress of the ONNX community.
What is ONNX?
ONNX is an open standard for machine learning model interoperability. According to the web site, ONNX defines a common set of operators — the building blocks of ML and deep learning models — and a common file format to enable AI developers to use models with a variety of frameworks, tools, runtimes, and compilers. It was originally open-sourced by Facebook and Microsoft in the Fall of 2017, and is now widely popular in the ML community. ONNX provides an intermediate representation for ML models and pipelines that makes it easier to exchange models between different frameworks. In addition, hardware manufacturers like Intel, NVIDIA, Qualcomm, and Habana like to have a single representation for which to optimize their code rather than dealing with a dozen different deep learning frameworks. Recently, ONNX was accepted as a graduated project to the LF AI Foundation.
ONNX Community meeting
There is a large community working on further developing the ONNX standard. In addition to regular conference calls of various subgroups, normally there are about three full-day in-person meetings per year where the developers get together to discuss our progress and next steps. Last year I attended two of them in Silicon Valley: hosted by Intel in March and by NVIDIA in August. The third community meeting was held in Shanghai in November, hosted by Microsoft. IBM was going to host the Spring 2020 community meeting but the plans were adjusted due to the global COVID-19 pandemic. We successfully hosted the meeting virtually on April 9. My colleague Thomas Truong worked with the ONNX Steering Committee and LF AI to create a great program for the meeting. In order to accommodate more time zones, it had to be reduced to three hours in length. The number of people attending at least parts of the meeting was 183, and many more watched the recordings later. All presentations and video recordings of the meeting can be found on the LF AI wiki.
During the introductions Harry Kim from Intel showed some facts about the growth of ONNX popularity during the last year:
We also learned about some new community members, pictured in the image below.
Zhipeng Huang presented on MindSpore, shown in the upper right corner of the diagram. It is an open source ML framework released by Huawei.
Emma Ning from Microsoft talked about performance breakthrough her team achieved with ONNX Runtime, an open source project started at Microsoft and closely related to ONNX.
Kishwar Shafin from UCSC told us how the use of ONNX and ONNX Runtime in the analysis of genomic data helped to make some important computation-intensive processing from 6 to 10 times faster.
Shounak Mitra and Tinga Su from Mathworks told us how MATLAB can import most ONNX models. I have used MATLAB in graduate school a very long time ago and was impressed by how far it has progressed since then.
Special Interest Groups
After a break the Special Interest Groups (SIG) and the ONNX Training working group delivered their reports.
My colleague Chin Huang is leading the Converters SIG together with Guenther Schmuelling from Microsoft. The converters provide tools for exporting ONNX from various ML frameworks and for importing ONNX models into frameworks for inference. Chin and his group are working on an ONNX to TensorFlow converter. Other colleagues at IBM are working on an ONNX to MLIR converter that will empower efficient model inference on a variety of computer hardware.
Vinitra Swamy is now leading the SIG responsible for ONNX Model Zoo and tutorials. Those assets help newcomers to learn how to use ONNX. I believe there are ways to help with the Model Zoo and tutorials if you want to start contributing. The SIG also keeps track of upcoming conferences and other public events where ONNX will be presented. If you are presenting on ONNX somewhere, create a pull request to update the Community Events page on GitHub.
ONNX Training Support
Several SIG reports mentioned ONNX Training, which is the main (preview) feature of the recently finalized ONNX release 1.7 and touches on most aspects of the standard.
ONNX Training support is a pivotal change for ONNX because it was originally designed for exchanging fully trained models only. Why would anyone want to use it for training? It turns out that there is substantial interest in this new feature. In some cases model training is a part of model deployment, when the model needs to be fine-tuned or updated with new data after deployment. Also, people want to be able to create a training spec or a partially trained deep learning model in one framework, then export that in ONNX, and later continue training in another framework or in ONNX Runtime. They also want to get consistent training results regardless of the specific hardware used for training. This has been in the works for over a year now. The task is not easy, due to the complexity of deep learning training and the fact that ONNX design prevents modifying parts of an ONNX model, while during training we need to update the weights of a neural network. Wei-Sheng Chin from Microsoft created a proposal that seems to work despite all those difficulties. The details of the proposal’s implementation were discussed on Gitter and in several GitHub pull requests, as well as the working group and SIG meetings.
Now the next step for ONNX Training implementation is adding new information into most ONNX operator specifications to help with implementing the Gradient operator for all ops. Then we will add some new utility functions into ONNX core and work with converter teams to implement export and import for training models.
ONNX vs PMML
During the meeting somebody asked how ONNX compares to Predictive Model Markup Language (PMML.) Since I have worked with PMML for about 19 years, I was able to answer this question. PMML is an open standard for representing predictive models in XML that started in the 1990’s and is pretty widely used nowadays. It is designed for traditional machine learning and does not support deep learning, at least not yet. PMML 4.4 includes 17 model types and a very rich set of data transformations, including built-in and user-defined functions, as well as ways to combine models together using ensembles and model compositions. ONNX was originally designed for deep learning, and the “traditional” part added later as ONNX ML. ONNX does not yet allow user-defined functions, and is using binary (protobuf) format for model representation as opposed to human-readable XML format of PMML. The binary format is definitely better for deep learning models, due to their size, but the human-readable PMML provides more transparency on the model.
We use PMML widely inside several IBM products and allow users to export PMML from many models they build. In addition to scoring (inference), PMML is very helpful for model visualization, as it includes ModelStats and ModelExplanation elements containing information helpful for model interpretation. I don’t think ONNX has similar features. You can learn more about my work in my blog, “State of the Art Model Deployment.”
In conclusion, ONNX is a very fast growing open standard, backed by many large and small corporations and individual contributors. It is the place to be, the open source project to work on for any developer interested in ML.
