[Background]
Let’s say my task is to develop some object detection/Image classification solutions on QNX Platform.For Example,Occupant Detection in car,Driver Monitoring system,activity detection etc. How to go about it?Which Deep Learning Framework to use?
Limitations:
- QNX officially does not support Tensorflow, Pytorch or any other famous Deep Learning framework.
- Each package in QNX is charged separately & it’s difficult to buy for individual developer or company.
- QNX has support of OpenCV 2.7& Matlab.And both has it’s own deep learning framework. But deep learning is supported from OpenCV3.0.
Right now the question is how we can develop object detection/image classification solution on QNX platform.Remember,building Tensorflow or Pytorch on QNX Platform is not that easy as all of them have dependency on many other libraries.Officially,Tensorflow is not yet supported on QNX.If it will be supported in future,then you or your company needs to pay extra amount for it.Then what’s the solution?
Which Deep Learning Framework to use in QNX Platform?
We need to find a deep learning framework which has less dependency.I found that Darknet Framework is the right choice. It’s because it is written in C & optionally, it’s dependent on OpenCV & CUDA. You can easily build Darknet Deep Learning Framework on QNX platform.
Overview on Darknet Framework
Darknet is a configuration based deep learning framework.You do not need to write the neural network architecture by using any programming language.Programmers only need to modify the configuration file to train , model & generate model.
Darknet is hosted at https://pjreddie.com/darknet/. Other famous open source is https://github.com/pprp/darknet-AlexeyAB.
Darknet support following features:
- Object Classification.
- Object Detection through YOLO v1/v2/v3.(Other detection algorithms such as SSD, RetinaNet etc. are not supported)
- RNN (LSTM, GRU & Naive RNN only supported)
- Style transfer
- Image Segmentation
[Building Darknet Framework on QNX Platform]
1.Set the QNX tool chain by executing qnxsdp-env.sh
2. Download git clone https://github.com/achalshah20/darknet-cpp.git
3. Modified CmakeList.txt to disable CUDA & OpenCV(Please check the CMakeList.txt files)
4. place the qnx7_aarch64.cmake file
3. mkdir build_qnx
4. cd build_qnx
5. ../darknet-cpp/build_qnx$ cmake — verbose=1 -G “Unix Makefiles” -DCMAKE_TOOLCHAIN_FILE=../qnx7_aarch64.cmake ..
6. make
(Note: This just to show how to build darknet framework in QNX platform using CMake tool.If you want to build other version of Darknet on QNX platform,then please try on your own. )
[Architecture]
Deep learning pipeline can be broadly divided in to
- Input data pipeline
- Defining Model Architecture
- Training
- Debugging, Profiling & visualization
- Performance Optimization
- Deployment
Input data pipeline
[Defining Model Architecture]:
In any neural network,we should define
- Layers : convolution, max pooling, batch normalization,dropout,skip-connection, softmax etc.
[convolutional]
filters=16
size=3
stride=1
pad=1
activation=leaky[maxpool]
size=2
stride=2
- Activation Function : logistic, relu, elu, tanh, leaky,linear etc.
- Optimization/Learning Algorithm: By default it uses Adam
- Cost Function: SSE(Sum Squared Error),Smooth,L1 etc.
[cost]
type=sse
- Anchor boxes (used for object detection)
In Darknet, model architecture is defined in the configuration file.Darknet supports all most all the layers such as Convolution,dropout,fully connected layer etc.You can check all the layers at https://github.com/pjreddie/darknet/blob/master/src/parser.c . If you are using some layers in your neural network,but they are not part of darknet,then you can not use your neural network architecture in darknet framework.Please refer the below link to know more about how to write configuration file of your neural network in darknet framework.
[Training]:
Hyper parameters(such as learning rate, momentum, decay rate, policy,batch size etc.) are defined in configuration file under net block.
[Debugging, Profiling & Visualization]
Base version of darknet only have “ — visualize” option to see the CNN layers.There are few contributors who added layer visualization on top of Darknet base version.Please check https://github.com/schwittlick/ofxDarknet/tree/master/.
[Performance Optimization]
- Optimization in only CPU Configuration:
NNPACK is an acceleration package for neural network computations.
NNPACK aims to provide high-performance implementations of convnet
layers for multi-core CPUs.NNPACK is not intended to be directly used by machine learning researchers; instead it provides low-level performance primitives to be leveraged by higher-level frameworks, such as Caffe, Torch, MXNet, Theano, Tensorflow, and Mocha.jl.It’s hosted at :
https://github.com/shizukachan/darknet-nnpack.
https://github.com/digitalbrain79/darknet-nnpack.
2. Optimization in Neural Network using Quantization Technique:
Quantization refers to the process of reducing the number of bits that represent a number. In the context of deep learning, the predominant numerical format used for research and for deployment has so far been 32-bit floating point, or FP32. However, the desire for reduced bandwidth and compute requirements of deep learning models has driven research into using lower-precision numerical formats. It has been extensively demonstrated that weights and activations can be represented using 8-bit integers (or INT8) without incurring significant loss in accuracy. The use of even lower bit-widths, such as 4/2/1-bits, is an active field of research that has also shown great progress.
The more obvious benefit from quantization is significantly reduced bandwidth and storage. For instance, using INT8 for weights and activations consumes 4x less overall bandwidth compared to FP32.
Additionally integer compute is faster than floating point compute. It is also much more area and energy efficient.
Quantization support in Darknet is hosted at https://github.com/AlexeyAB/yolo2_light#yolo2_light.Please refer this link for for more information.
[Deployment]
Darknet model can be converted in to tensorflow model by using darkflow tool.After conversion we can deploy the tf model directly on the target or we can use tensorflow serving feature.If on the target platform OpenCV is supported,then we can use it’s feature to load darknet model & do the prediction.
Conclusion:
As you can see base version of darknet,https://pjreddie.com/darknet/, does not support features such as quantization,visualization,NNPACK etc.But https://github.com/pprp/darknet-AlexeyAB supports basic features which are required for Object Detection & Image Classification problem.If we can integrate NNPACK & Visualization features in to AlexeyAB repo, then we can use it in QNX platform for object detection & image classification problems.
Check my other articles:
- https://medium.com/@satya15july_11937/object-detection-with-transformer-based-architecture-cbcb6b9fa5fe.
- https://medium.com/@satya15july_11937/network-optimization-with-quantization-8-bit-vs-1-bit-af2fd716fcae.
Reach me at:
- LinkedIn: www.linkedin.com/in/satya1507
- Mail-Id: satya15july@outlook.com
