Alias free resize with RenderScript
Resizing a bitmap is a quite common task. Let’s say you want to create a thumbnail from a source image; you will have to resize it to quite a smaller size. However, if you try to do that using the provided tools, you’ll end up with aliasing artifacts.
Android’s Bitmap class provides many methods that can downscale an image. Under the hood, it uses a Canvas and a Paint object that has bilinear filtering enabled. However, when scaling down an image, bilinear filtering is not enough to prevent aliasing.
In the images below, an image with `2880 x 2160` resolution was resized to `360 x 270`. In the top one, you can see the result of using Bitmap’s CreateScaledBitmap() method and on the bottom one you can see a proper resizing result. The aliasing artifacts due to subsampling are quite intense on the top one and the result is unacceptable.
For example the following code creates an image having 1/8th size to the original:
BitmapFactory.Options options = new BitmapFactory.Options();
options.inSampleSize = 8;
Bitmap bitmap = BitmapFactory.decodeFile(imageFile.getAbsolutePath(), options);
One major problem with this approach is that if your bitmap is already decoded, you can’t use this method. Also, sample size can only use values that are powers of 2. This means that you can’t downscale to an arbitrary resolution, but only to halves of the original resolution. Last but not least, this method works great with JPEG files, but if the decoded image is of PNG type you end up with aliasing.
A workaround to manage downscaling to an arbitrary size would be to first decode the image to the next power of 2 resolution and then use the
Bitmap methods to downscale it exactly to the size you want. However, the result will be slightly blurry. Update: Instead of using the
Bitmap methods to downscale the image to arbitrary size after the
decode method, you can achieve the same visual result from within the
decode method by also specifying the
inTargetDensity as explained in this video.
A workaround to getting high quality downscaling to powers of 2 dimensions when the bitmap is already decoded (and we can’t use the
BitmapFactory’s decode methods), is by successively resizing the bitmap to half dimensions. You can see two implementations in this gist. Note that if you need to downscale to arbitrary size, one more resize pass is needed to get from the next higher power-of-2 dimensions to the desired dimensions. As mentioned previously, this produces a bit blurry result.
Prefiltering with RenderScript to the rescue
As explained in all signal processing books, in order to subsample a signal and avoid aliasing, you first have to prefilter the signal to exclude high frequencies. Then, you can subsample it.
In other words, we first have to “blur” the image and then subsample it. The amount of blur we need to apply depends on the prefilering method and how much we need to subsample the image (see: Nyquist frequency).
One of the quickest, implementation-wise, and fastest, performace-wise, ways to blur an image in Android, is using RenderScript. Fortunately, RenderScript comes bundled with a Gaussian filter implementation, the ScriptIntrinsicBlur, that can apply the prefiltering for us. It also provides an intrinsic for resizing, the ScriptIntrinsicResize, which uses bicubic interpolation.
So, the idea is that we’ll first apply Gaussian blur to the image and then subsample it using bicubic interpolation.
Perhaps the most difficult part of the code is calculating the Gaussian’s radius. At first, I calculate the Gaussian’s sigma relative to the subsampling ratio as:
resizeRatio/p. This is derived by the fact that a Gaussian’s frequency response is another Gaussian and a bit of math.
ScriptIntrinsicBlur has a radius parameter though, not a sigma. But Google’s sourcecode shows the relationship between the two:
sigma = radius * 0.4 + 0.6. Solving for radius gives us:
float radius = 2.5f * sigma -1.5f;.
The rest of the code is self-explanatory. We create allocations for the source image, blurred image and output image. We first apply Gaussian blur and then resize it. We also do a bit of memory management.
Performance and comparison
I tried the proposed method to resize a 2880x2160 image to 640x480. I also tried the Bitmap.createScaledBitmap() method, the successive resize method (using multiple Bitmap.createScaledBitmap() calls) and the successive resize method using RenderScript’s bicubic interpolation. Here are the results:
The “createScaledBitmap” method, even though the fastest, has extreme aliasing. The “successive resizes” is a bit blurry and the “successive resizes (RenderScript)” is a bit sharper. The “proposed method” has by far the best quality.
If we downscale the image to the power-of-2 size of 360x270 we have the following results:
Because we are scaling down by a power-of-2, the results are a bit different quality-wise. The “successive resizes” method gives identical result to the proposed method due to the fact that no extra-resize is needed. The successive resizes (RenderScript)” is a bit sharper as usual.
The proposed method takes 380ms to scale a 5760x4320 image to 640x480.
All experiments ran on a Nexus 5.
Note that if
resizeRatio is larger than 1, then we are actually upscaling and prefiltering with Gaussian filter isn’t needed.
Gaussian filter is not the best prefiltering method, but the reason it was chosen here is due to the fact that RenderScript contains a ready-to-use intrinsic. Ideally, we could use an averaging filter, but this would require to write our own RenderScript kernel.
Applying a Gaussian blur to a large image is demanding. Also, the bigger the
resizeRatio is, the larger the Gaussian radius gets and thus the processing gets more computationally expensive. What we could do to improve performance, is use BitmapFactory’s decode method to subsample the image and then apply this algorithm to the resulting image. For example, if the source image has a width of 5760 and the target width is 640, the ratio is 9. We could decode the image using
options.inSampleSize = 2 and then apply the algorithm to the decoded (half size) image using
resizeRation = resizeRatio/options.inSampleSize, which in our case is: 9 / 2 = 4.5.
If you are interested in supporting older devices, you can use RenderScript via V8 Support Library. In my tests, the support library had better performance than the native RenderScript runtime!