Image Classification with MXNet Scala Inference API

Author: Qing Lan, Roshani Nagmote

With the recent release of MXNet version 1.2.0, the new MXNet Scala Inference API was released. This release focuses on optimizing the developer experience for inference applications written in Scala. Scala is a general-purpose programming language that supports both functional programming and a strong static type system, and is used with high scale distributed processing with platforms such as Apache Spark.

Now that you have been given the grand tour of the new Scala Inference API in the first post of the series, you’re ready to try it out yourself. You will first need to setup your dev environment with mxnet-full package, and then you can try your hand at an image classification example and an object detection example (which we will demonstrate in the next post).

Environment Setup (Linux/OSX)

Use maven to install the mxnet-full package. Add the dependency in the pom file. Please change <your_platform> to the platform you are currently using. (OSX: osx-x86_64-cpu, Linux: linux-x86_64-cpu/gpu)

<dependency>
  <groupId>org.apache.mxnet</groupId>
  <artifactId>mxnet-full_2.11-<your_platform></artifactId>
  <version>1.2.0</version>
</dependency>

If you are using IntelliJ, you should be able to see the package being imported. You may also follow this tutorial to setup IntelliJ with the MXNet Scala Package.

Image Classifier Example

In this section, you will use a pre-trained Image classifier model to do the inference. This example uses the ResNet152 model. You may use this script to download the example model files.

The following are the required libraries to rebuild this example.

import org.apache.mxnet.Shape                      
import org.apache.mxnet.{DType, DataDesc, Context}
import org.apache.mxnet.infer.ImageClassifier
import scala.collection.JavaConverters._
import java.io.File
import scala.collection.mutable.ListBuffer

Step 1: create main function to run the example

object ImageClassifierExample {
  private val logger = LoggerFactory.getLogger(classOf[ImageClassifierExample])
  def main(args: Array[String]): Unit = {
    val inst = new ImageClassifierExample
    val parser: CmdLineParser = new CmdLineParser(inst)
    val context = Context.cpu()
    ...

The context here is to define the model the code will be running. You can change this line to context.gpu() if you would like to run with GPU(s). For this example, we use this image.

    ...
    val modelPathPrefix = "absolute/path/to/your/model"
    val inputImagePath = "absolute/path/to/your/image"
    ...

Then you add the paths for models and a test image to use the Inference API.

Step 2: Load the model and do inference

The following code is a continuation from the previous code blocks:

    ...
    val dType = DType.Float32
    val inputShape = Shape(1, 3, 224, 224)
    val inputDescriptor = IndexedSeq(DataDesc("data", inputShape, dType, "NCHW"))
    ...

An inputDescriptor is required to define the input source and configuration for the model. “data” is the name of the input data. input shape is the shape of input image. The input matrix will be in three channel with 224 * 224 pixel sizes.

   ...
   val imgClassifier: ImageClassifier = new
        ImageClassifier(modelPathPrefix, inputDescriptor, context)
   val img = ImageClassifier.loadImageFromFile(inputImagePath)
   val output = imgClassifier.classifyImage(img, Some(5))
   ...

After we sort out all of the inputs, we create an Image Classifier object and use it to load the image. Then we start doing classification on that sample image. The “Some(5)” field means we will pick up top 5 prediction with the best accuracy. This field is optional and predictions with unsorted order will be the default option. Once we finish this step, we just need to show the output.

   ...
   for (i <- singleOutput(0)) {
           printf("Classes with top 5 probability = %s \n", i)
   }
}

Result

After you have finish the previous step, you should be able to see the output similar to the one shown below (here we used a pug dog image):

Classes with top 5 probability = (n02110958 pug, pug-dog,0.5032309) 
Classes with top 5 probability = (n13905792 wrinkle, furrow, crease, crinkle, seam, line,0.06664828) 
Classes with top 5 probability = (n02084732 pooch, doggie, doggy, barker, bow-wow,0.029262988) 
Classes with top 5 probability = (n02085374 toy dog, toy,0.028741771) 
Classes with top 5 probability = (n02083346 canine, canid,0.02784521)

The first element is the class “n02110958 pug, pug-dog” and the second value is the confidence of the class as determined by the classifier.

Conclusion

After this brief walk through, you should be able to create a Image Classifier using MXNet Scala Inference API. You can find more detailed information on the code for this example in the MXNet project repository’s Scala Infer Image Classifier example.

In our next blog post, we introduce handling Object Detection tasks using the MXNet Scala API. Check it out next!