JPEG 2000 issue in iOS Apps
We use images in our apps very often. Generally, they are .jpg, .png or .pdf images, but sometimes you can receive image of not so popular type (.jp2, for instance). What’s .jp2 image?
JPEG 2000 (JP2) is an image compression standard and coding system. It was developed from 1997 to 2000 by a Joint Photographic Experts Group committee chaired by Touradj Ebrahimi (later the JPEG president), with the intention of superseding their original discrete cosine transform (DCT) based JPEG standard (created in 1992) with a newly designed, wavelet-based method. The standardized filename extensionis .jp2 for ISO/IEC 15444–1 conforming files and .jpx for the extended part-2 specifications, published as ISO/IEC 15444–2.
While there is a modest increase in compression performance of JPEG 2000 compared to JPEG, the main advantage offered by JPEG 2000 is the significant flexibility of the codestream. The codestream obtained after compression of an image with JPEG 2000 is scalable in nature, meaning that it can be decoded in a number of ways; for instance, by truncating the codestream at any point, one may obtain a representation of the image at a lower resolution, or signal-to-noise ratio — see scalable compression. By ordering the codestream in various ways, applications can achieve significant performance increases. However, as a consequence of this flexibility, JPEG 2000 requires encoders/decoders that are complex and computationally demanding. Another difference, in comparison with JPEG, is in terms of visual artifacts: JPEG 2000 only produces ringing artifacts, manifested as blur and rings near edges in the image, while JPEG produces both ringing artifacts and ‘blocking’ artifacts, due to its 8×8 blocks. For more info visit: https://en.wikipedia.org/wiki/JPEG_2000
OK, now we know, that JPEG 2000 is especial image data type and it demands more computationally resources than general JPEG format. What does it mean for iOS developer? It means problems 🙌.
Let's make one small experiment. We will download large .jpg format image and create table view with this image inside every cell. For example, let’s take this NASA’s Cassini spacecraft’s image (CGF_STILL_00025.0.jpg). It is general JPEG image with size of 165 kb and 1820x1213 dimension.
Then I just converted this image to .jp2 image with the same dimension and size.
After that I added this image pair to demo project folder (not in assets because you can’t add .jp2 images as an asset). Take a look to the code below.
CassiniImagesViewController just take image of appropriate type from main Bundle and put it into every of 15 UITableView’s cells.
Now we are ready to test this small app 😈
And… If we are in normal JPEG mode, then all is OK. Scrolling works smoothly… but what if we will toggle our app to the JPEG 2000 mode? Our iPhone will behave noticeably worse. There will be a lot of freezes during scrolling.
OK, let’s measure .jp2 file influence on rendering process. Look at this tool.
Core Animation allow you to measure graphics performance on your real device. So… Just compare data below!
Note: I took measurements on iPhone 7 with iOS 13.1.1.
If app is in JPEG (normal .jpg file) mode, then we have 27–54 FPS in scrolling list of Cassini.jpg images, but if it’s in JPEG 2000 (.jp2 file) mode, then our device can provide us only 3–37 FPS and it’s the reason why we observe a lot of freezes in UI.
But let’s look at GPU hardware utilization. For JPEG it is from 7.0% up to 13.0%, and for JPEG2000 it’s placed in range 0.0–8.0%.
Based on this, I can assume that the problem lies in GPU using during .jp2 image rendering. More precisely, iOS isn’t using the GPU in JPEG 2000 images rendering. But does it mean that it should use CPU instead? Let’s check it!
For this purpose I used Activity Monitor tool.
As I expected iOS really uses CPU instead of GPU for JPEG 2000 images rendering. And it leads to interface freezes, because CPU is not best solution for rendering graphics.
But what can you do if your app gets .jp2 images from API (for instance)?
I have only one answer for you. It’s needed to convert .jp2 image to normal .jpg. For this purpose you can use something like this:
Thank you for reading! If I made a mistake, please, let me know!
Test project you can find here: https://github.com/AleksandrRuzmanov/JPEG2000_IssueDemo.git