With more and more businesses looking to scale up their operations, it has become integral for them to imbibe both machine learning as well as predictive analytics. AI coupled with the right deep learning framework has truly amplified the overall scale of what businesses can achieve and obtain within their domains.
As you might already know, we develop a decentralized open platform for virtual assistants. Creating any virtual assistant is a process that is directly related to training the neural networks. Thanks to our great data scientists and developers, we created our own software which is not worse than the Google’s one.
My name is Andrey Kurganov, I’m one of the developers of PuzzleLib and here I would like to reveal the insights into our know-how.
What is PuzzleLib?
PuzzleLib is a Deep Learning framework: a software library that allows to construct a neural network — allows the user to, makes it possible to visualize its architecture, train it and export it for use in mobile or server applications.
Analogs: TensorFlow by Google, Torch by Facebook, CNTK by Microsoft.
What is important for a framework?
- Algorithms: the library must support classical neural algorithms implemented as computation modules.
- CPU and GPU support: training is better with GPUs, but CPU support can be crucial when it is not possible to use expensive GPUs.
- Cross-platform: it is crucial to have an ability to use Deep Learning on various operation systems and CPUs, e.g. on mobile ones.
- Speed: training a neural network for Big Data tasks can take days. 1.5–2 times acceleration is significant for the workflow. In addition, fast data processing is important in real time applications (e.g. video analysis on board).
PuzzleLib advantages
- Speed comparable to Facebook Torch. We achieve highest speed because we use only low-level computation NVIDIA libraries and effective processor resources usage.
- We support CPU and GPU inference and training.
- Intel, ARM, Baikal, NVIDIA support.
- The framework supports all most popular operating systems: Linux family, Windows, Mac OS, iOS and Android
- We implemented an automatic computation optimization technique. In some cases it doubles the speed of network training.
- More than 60 neural modules were implemented, which allow to build convolutional and recurrent neural networks, and nets with custom computational graphs. The list of supported modules can be found below.
- The learning curve for PuzzleLib is much shorter.
- PuzzleLib is our own patented IP.
PuzzleLib modules
Any neural network is a computational graph: every node is a PuzzleLib module that performs specific operations with the data provided. Below goes the list of currently supported modules.The modules make it possible to construct any modern Deep Learning architecture. The list of currently supported modules is provided below:
Basic modules:
- Activation (Sigmoid, Tanh, ReLU, ELU, LeakyReLU, SoftMaxPlus)
- AvgPool (1D, 2D, 3D)
- BatchNorm (1D, 2D, 3D, ND)
- Conv (1D, 2D, 3D, ND)
- CrossMapLRN
- Deconv (1D, 2D, 3D, ND)
- Dropout (1D, 2D)
- GroupLinear
- InstanceNorm2D
- KMaxPool
- LCN
- Linear
- LRN
- MapLRN
- MaxPool (1D, 2D, 3D)
- MaxUnpool2D
- Pool (1D, 2D, 3D)
- RNN
- SoftMax
- SubtractNorm
- Upsample (2D, 3D)
Special modules:
- Embedder
- SpatialTf
Supplementary modules:
- Add
- Concat
- DepthConcat
- Flatten
- Glue
- Identity
- MoveAxis
- Mul
- MulAddConst
- NoiseInjector
- Penalty
- Replicate
- Reshape
- Split
- Sum
- SwapAxes
- Tile
- ToList
- Transpose
Optimization algorithms implemented in PuzzleLib
To train a neural network, it is necessary to iteratively update its weights using an algorithm called “optimizer”. Different optimizers are needed for different tasks (for example, regression, texts or image classification).
- AdaDelta
- AdaGrad
- Adam
- Hooks
- LBFGS
- MomentumSGD
- NesterovSGD
- RMSProp
- RMSPropGraves
- SGD
- SMORMS3
We have already implemented several commercial projects based on PuzzleLib for different enterprises. These projects are related to the processing of photo and video information. In addition, the framework allows you to perform automatic classification of texts by subject or key, as well as to assess the proximity of texts and perform machine translation.
If you have any questions or would like to support our project, please contact us at hello@sova.ai.
Visit SOVA official website…
OR chat with us in Telegram.
Don’t forget to clap! I tried hard to write this article, praise me a little for that :-)
