Introducing DL4JRA — Low Code Deep Learning 4 Java
Malaysian Smart Factory 4.0 @ SHRDC recently developed an open source low code deep learning application using Deeplearning4J library called DL4JRA (Deeplearning4Java React App). As someone who learnt DL4J library, I had to try the app out, to see if it aligns with what I learnt. I’m not proficient with the library per say, but I’m going to try it out with the knowledge that I know from it.
Quick introduction about me (so you would understand the context of my knowledge level); I am a Mechanical Engineering graduate with basic programming knowledge who decides to jump into Machine Learning and Data Analytics. I joined an AI training program which taught DL4J so my knowledge on Java is limited to only basic DL4J knowledge.
Continue on. According to their GitHub page, the purpose of developing DL4JRA is to increase the adoption of utilizing Artificial Intelligence (AI) in various use cases and applications within the wider community through low code development. Low Code application strategy of utilizing DL4JRA would allow the users to effectively generate datasets, label (using external labelling tools such as Label Studio), train, and deploy within a development cycle. This allows the user to perform rapid application development and prototyping of relevant deep learning applications for computer vision applications using the DL4J framework. This lowers the barrier of entry for the community to be able to be exposed to utilizing AI for their application(s) using DL4J.
Introduction
Eclipse Deeplearning4Java is a programming library written in Java for the Java virtual machine (JVM), developed mainly by machine learning group in San Francisco and commercially supported by Skymind which bundles Deeplearning4J(DL4J), TensorFlow, Keras and other deep learning libraries in an enterprise distribution called the Skymind Intelligence Layer. Deeplearning4j was contributed to the Eclipse Foundation in October 2017. It is an open-sourced and actively developed Deep Learning Software, according to Wikipedia.
Let’s jump into the app. Watch this tutorial video that I have prepared on how to install the app.
Contents of DL4JRA
When you first open DL4JRA, you will see 5 functions which are:
· Generate Dataset
· Neural Network
· Object Detection
· Image Classification
· MQTT Signalling
Generate Dataset
If you don’t have image datasets prepared, don’t worry! You can generate a number of image datasets with the inbuilt function using your webcam. I was surprised that this app is included with the function to take a picture of your own dataset and build your custom dataset. You just need the object that you want your model to train on and put it on your webcam view. Then, click screenshot and voila, you have a picture of your object!
You can train your model with only a few pictures of course, but it is much better if you have a lot of pictures (I’m talking hundreds and thousands!). So, for that, you can specify the count of your screenshots and you will have almost identical shots of your object. I tested it out by moving the object around in front of the webcam to give some variation on the dataset, and DL4JRA are able to take screenshots according to your real-time object movement.
I usually make my dataset by recording videos and use another program to split the video by frames, so for the app to automatically take a number of screenshots on real-time movement made it easier to build your custom dataset.
Neural Network
This is where most of the DL4J library takes place. I see that there are 4 tasks that can be done within this function which are:
· Object Classification (Using Multilayer Network)
MultilayerNetwork is the simplest and a basic way to start making your Convolutional Neural Network in DL4J. You can use Multilayer to make your image classification network or CSV classification network. DL4JRA provides a few layers you can use to suit your needs, Convolutional, Subsampling, Dense and Local Response Normalization.
If you don’t wish to create your own CNN, DL4JRA also provides some pre-existing models available for you to use, which are SqueezeNet, VGG16 and VGG19. CNN usually consists of many nodes and layers, so these models were packed with the nodes and optimization which would be difficult to manually arrange with the nodes. SqueezeNet is the lightest model. The VGG models are heavier and if you have a strong CPU and RAM storage, then you can use these models to train. Else, it won’t work due to insufficient memory. I have 8GB RAM and I can only run SqueezeNet. You would need more than that to run VGG16 and VGG19.
· ComputationGraph Network
Everything that MultiLayerNetwork can do, ComputationGraph can do as well — though the configuration may be a little more complicated. I admit, I am not proficient in the usage of ComputationGraph except to make Recurrent Neural Network (RNN) and Long-Short Term Memory (LSTM) to do CSV TimeSeries classification, and that is what DL4JRA can do with the ComputationGraph Network nodes.
· Image Segmentation
Image Segmentation is a process to subdivide an image into its constituent parts or objects. When I try out this function, a new window will pop-up, showing my segmentation results, which I find really nifty.
· Object Detection (Using Pretrained Model)
When I first dabbled in Machine Learning, I learnt about the YOLO models. I find it exciting when I know we can use YOLO models within the DL4J library. DL4JRA have prepared specialized nodes to train your object detection datasets using YOLO format (by YOLO format, I mean having annotation files, containing the object labels in the images). The existing pretrained models in DL4JRA are TinyYOLO and YOLOv2. You can use this to train your model to be able to detect the objects that you want to be detected using pretrained model TinyYOLO and YOLOv2 OR retrain your existing custom TinyYOLO model.
So far, it is very easy to use and navigate. This part of the app use nodes to arrange your neural network flow (if you have experienced using Node-RED, the usage is similar to that). So, you’ll just have to DRAG-&-DROP.
Object Detection
This is the fun part, in my opinion. After you trained your model, of course, you would want to test it out. This function lets you test out the model that we have made, or if you don’t have a model yet and just want to try around, you can also use the available models TinyYOLO and YOLOv2.
There are also recording and logging functions if you wish to see the result afterwards which I find good and can compensate for the small detection window and lack of confidence percentage label. I can come back to the object detection recording and watch the detections properly since the detection window is a bit small and I can’t see the detected labels properly; and also check the confidence percentage by checking the log.
Image Classification
You can also test your image classification model here. I find it weird that the app has an object detection function and image classification function but after using it, I can see that there are some stark differences. It is undeniably similar to Object Detection, but instead of using YOLO and show detection box; this runs your CNN model in the background and classifies the image as a whole. You can put your object in the webcam view and see what your model classifies the image as. You’ll need to configure the label names alphabetically though, or else, it will only show Label-1, Label-2 and so on. We can only test using the model we trained and no pre-existing model to test with, unlike in Object Detection testing.
MQTT Signalling
Finally, the last function to introduce is the MQTT signalling function. I don’t know what MQTT is and what can I do with it but I still want to test it out for the sake of the article.
So, what is MQTT? It is a simple messaging protocol, designed for constrained devices with low bandwidth. So, you can use this to connect DL4JRA with other IOT devices. The protocol they used is EMQX which is an open-source MQTT broker but there is one part of the MQTT that I need to install a local broker to view the detection output from Node-Red. This would be a great option if you would like to deploy on a local server as compared to using a public broker where our security might be a concern.
I tried out Node-Red and tested out the MQTT function and it was good. I can see my detection results from another computer, thanks to MQTT signalling. I think this function would benefit those who want to run and view their progress from another machine.
And that briefly sums it up for DL4JRA introduction!
I have prepared tutorial videos that are available in this playlist.
Here are the initial developers and contributors of DL4JRA:
Li Xianlong(LinkedIn)
Tam Wei Yao (LinkedIn)
Nicholas (GitHub)
Nurul Izzati Suhardi (LinkedIn)
Malaysian Smart Factory 4.0 @ SHRDC is open to work with the community of developers, universities, and industries to contribute to the project! You can contact Ts. Dr. Chua Wen Shyan to discuss on the use cases and contributions for DL4JRA.
You can visit and clone the react app repository through their DL4JRA GitHub page.
