Faster Image Processing in Android

Published in

Bobble Engineering

5 min readMar 26, 2021

While working on an android app we are sure to encounter Image Processing in one way or other. This article talks about how to process Bitmaps. we will be starting will basic methods, and then move towards efficiency. Before we start with different ways to process Bitmaps, lets see some basic Mistakes we do while loading Bitmaps -

Rendering Actual Image Size — We use the actual size of the image for rendering rather than the display size of the device on which it is being shown. it takes a lot of time.
Loading Bitmaps Synchronously — Its not a wise idea to load your Bitmaps synchronously. it blocks your UI, and hangs your application if you are loading multiple Bitmaps or loading large size bitmaps.

Now that we know what to avoid while loading bitmaps, lets quickly get started with what options do we have to process Bitmaps?

getPixel() and setPixel()

This is basic method of getting each pixel of the Bitmap and then modifying then with the required value and then setting them back. it looks something like

val bitmapCopy = bitmap.copy(Bitmap.config.ARGB_8888,true)
   for (i in 0 until bitmap.width){
      for (j in 0 until bitmap.width){
         bitmapCopy.setpixel(i,j,Color.RED)
      }
   }
bitmap = bitmapCopy

The above code will iterate the bitmap line by line and will modify the bitmap pixels to the desired formatting. this looks simple, however there are few Disadvantages for the above code.

This is super slow.
This runs on the CPU.
Absence of parallel Execution.

Image Library Transformations

This Include inbuild transformations in the image loading library like Glide, GPUImage, Fresco etc.

// Glide
Glide.with(this)
   .load(url)
   .apply(bitmapTransform(new BlurTransformation(25,4)))
   .into(img_view)//GPUImage
gpuImg = GPUImage(this)
gpuImg.setGLSurfaceView(surfaceView)
gpuImg.setImage(imgUri)
gpuImg.setFilter(new GPUImageSepiaFilter())

As you can see from above this is pretty simple to use, It is fast as well. there is a huge disadvantage of it is you cant achieve everything with it.

ColorMatrixColorFilter

This method uses the color matrix for adding color filters on the bitmap. we use Canvas and Paint to draw a new bitmap using Matrix filter for color filteration.

val canvas = Canvas(bitmap)
val paint = Paint()
paint.colorFilter = ColorMatrixColorFilter(
      ColorMatrix(floatingArrayOf(
      -1f,0f,0f,0f,255f,
       0f,-1f,0f,0f,255f,
       0f,0f,-1f,0f,255f,
       0f,0f,0f,-1f,0f
      )))
canvas.drawBitmap(original,0,0,paint)

It basically modify all the pixel values using the matrix . It works really fast and is easy to use. it can be used to do a single operation on all the pixel easily, and it runs on the GPU. however we still cant operate on individual pixels.

RenderScript

Renderscript is a framework on android for computationally expensive tasks. It runs parallel tasks on CPU or/and GPU depending on the device. It uses a C99 derived custom language for writing high performance computation code. To Use renderscript we create a *.rs file and write compute code in that

//render.rs
#pragma version(1)
#pragma rs java_package_name(app.touchtalent.renderscript)
#pragma rs_fp_fullstatic float bright = 0.f;//Function to set brightness value of the image
void(setBright(float b){
   bright = 255.f / (255.f - b);
}//Function to compute brightness value
void exposure(const uchar4 *in, uchar4 *out){
   out->r = clamp((int)(bright * in->r), 0, 255);
   out->g = clamp((int)(bright * in->g), 0, 255);
   out->b = clamp((int)(bright * in->b), 0, 255);
}

Kotlin Class

class Brightness(val context:Context){
   val renderScript = RenderScript.create(context)
   val script = ScriptC_exposure(renderScript)fun brighten(brightness:Float, image:Bitmap):Bitmap{
      val outputBitmap = Bitmap.createBitmap(image)
      val tmpIn = Allocation.createFromBitmap(renderScript,image)
      val tmpOut = Allocation.createFromBitmap(renderScript,outputBitmap)
      
      script.invoke_setBright(brightness)
      script.forEach_exposure(tmpIn, tmpOut)tmpOut.copyTo(outputBitmap)
      tmpIn.destroy()
      tmpOut.destroy()
      return outputBitmap
   }
}

As you can see from above, there is a lot of code as compared to Java, the language is not very easy to understand, and we are managing the memory ourself, however there are big advantage of renderscript

Its 8~10 times faster compared to java Code
It is highly customizable, we can operate on pixels individually
The execution is parallel and is done on the GPU
The intractions between the layers is not very complex

OpenGL

It uses Open GL Library. which is supported by android both by framwork API and native development kit. It uses two classes GLSurfaceView which is a View where you can draw and manipulate objects using openGL API calls. It can be used by adding Renderer to it.
Other class is is GLSurfaceView.Renderer It defines the methods required for drawing graphics in a GLSurfaceView .

More on OpenGL Later, however it is sufficient to know that it is fast and high performance library.

Native (C++)

We can use C++ using NDK to write highly efficient code and Native Access(JNA) library to call native code from Java side

C++ function to convert pixels to greyscale.

void toGrayScale(int *pixels, int len) {
     for (int i = 0; i < len; i++) {
          //getting individual color values from each pixel
         int A = (pixels[i] >> 24) & 0xFF;
         int R = (pixels[i] >> 16) & 0xFF;
         int G = (pixels[i] >> 8) & 0xFF;
         int B = pixels[i] & 0xFF;
          //averaging Red, Green and Blue value to get gray scale value
         int gray = (R + G + B) / 3;
          //assign same gray value to Red, Green and Blue.
         //alpha value is unchanged
         pixels[i] = (A << 24) | (gray << 16) | (gray << 8) | gray;     } }

This is called from java using JNA. complete guide on this is coming soon.

Native methods are more efficient and are faster also they allow us individual pixel modification. however they are more complex, requires time, and are difficult to debug.