Huge Images, Small Phone

Efficient Large Image Loading on iOS

Scroll views with images are featured ubiquitously in iOS apps. Images help engage users but showcasing them on a small device such as the iPhone comes with performance challenges. Directly loading large images incurs a high memory cost and limits performance. It would be great if we could conserve memory by showing only areas of the image the user has zoomed into; and if zoomed out, show a lower resolution version of the image.

Image credit: NASA/JPL-Caltech

As it turns out, the Core Animation framework offers a built-in solution called CATiledLayer. Showing tiles of an image in higher resolution rather than the entire image means that only parts of it are loaded into memory at once, resulting in faster load time and lower memory usage. To see tiles in action, zoom in on an area in the Maps app and see the zoomed-in area sharpen after a short wait. This post will show how to load large images efficiently using tiles.

This article will cover:

1) Implementing a scroll view backed by a CATiledLayer.

2) Cutting and caching image tiles using CATiledLayer, introduced In Apple’s WWDC 2010 video (Designing Apps with Scroll Views).

Implementing a scroll view backed by a CATiledLayer

To implement a scroll view that uses CATiledLayer, we must first create a custom tile view. This view will be responsible for requesting the parts of the image that we need to show and drawing them. You can read more about the properties of CATiledLayer here.

1. First, we implement a custom view and override its layer class to use CATiledLayer.
2. Content scale factor defines the relationship between points and pixels (1:1 for non-retina displays, 1:2 for retina). We always return a content scale factor of 1 since we will be drawing images directly.
3. We inject a tileManager into the tiling view. This tileManager will be responsible for providing the image sections that the tiling view requests. We will cover its core functionality in the next section.
4. We override drawRect(_rect: CGRect) which will be called by the scroll view when the user zooms in on an image. When drawRect is called, we can access its rect property to determine the tiles we need to supply.

Next, we implement drawRect(_rect: CGRect):

There’s a lot of math here but essentially we calculate the frames of the tiles we need and ask the tile manager for the part of the image that maps to the tile. We then draw each tile using its respective image. This code is adapted from the WWDC tutorial on CATiledLayer (WWDC 2010 video Designing Apps with Scroll Views).

1. We get the current scale of the view and apply that same scale to the tile size width and height.
2. Calculate the number of columns and rows we will need to draw in the view. We ask the tile manager for the image tile that corresponds to the tile scale and dimensions.
3. We draw the image tile in the rect.

The final step is to add an instance of TilingView to a custom scroll view. We’ll also need to set the maximum and minimum zoom scales of the scroll view depending on the image’s aspect ratio.

Cutting and Caching Image Tiles

At Frame.io, our API returns both high and low-resolution images. We use a download task to download the resource from the high-resolution URL. If successful, we move the downloaded resource to a path (called highResolutionImagePathURL below) in the local cache directory using the file manager. RayWenderlich.com has a great tutorial on download tasks and local caching.

In order to cut the image tiles we need, we first need to efficiently access the high-resolution image (remember calling UIImage(named:) will load it directly into memory).

Now that we have a file location path reference, we can access it using CGDataProvider and CGImage (using jpegDataProviderSource). This will return a CGImage.

Next, we use the tile manager to cut and cache the image tiles:

1. First, we need to figure out the corresponding rect on the image. The mappedImage represents the image we want to display depending on the zoom scale. For example, we may choose to only show the high-resolution image if the resolution is above 4000. Otherwise, we show a smaller image.
2. We crop the image for the tile rect and use FileManager to save the tile. Remember to clear the cache when dismissing the scroll view.

We recently launched our Image Review feature at Frame.io that utilizes CATiledLayer to enable zooming, panning, and annotating on high-resolution images on the iPhone. We found that CATiledLayer solves the performance problems that result from loading high-resolution images by conserving memory usage without limiting user interaction. Below is a side-by-side memory usage comparison of loading images with and without CATiledLayer (“Memory Report” is accessible from the debug navigator):

The non-tile image loader used almost 8x as much memory as the tile image loader! The difference may not be noticeable on an iPhone simulator on the laptop, but it is significant on a small device like the iPhone and a high memory pressure will likely crash the app or hinder performance. So the next time you need to load large images, consider using CATiledLayer.