Reducing Memory Footprint and OOM Terminations in iOS
How to avoid getting killed by the OS.
In this article, we are going to explore some techniques to make applications consume less memory (RAM) and to prolong their life expectancy.
First, we will describe some key concepts such as OOM terminations and memory warnings. Then we will describe different techniques to decrease the memory usage and finally, we’re going to discuss how to deal with memory warnings. This ended up being a little bit longer than I anticipated, so here’s a table of contents:
Important concepts:
Techniques for a lower memory footprint.
- Color Profile.
- UIGraphicsRenderer from Image I/O.
- UIGraphicsRenderer for composite images.
- Load images incrementally using Image/IO.
- Downsampling with Core Graphics.
- Download smaller images.
- Managing Caches.
- Unload unused sections of the app.
- Get rid of memory leaks.
What to do on memory warnings?
Some important concepts.
1. iOS kills applications when it needs memory.
Imagine that a user launches Instagram and sends it to the background to answer a phone call. Then she opens Twitter, Facebook, a game or any other app and finally the Music app. Each time iOS opens one of those apps, it might need to free memory to allow such application to run smoothly. At this moment, if she goes back to Instagram, she might find that it starts from scratch: it was terminated in between sessions.
This is a very common situation. If many apps that take up a lot of memory are opened, iOS might start killing apps. The chances of any app to be killed will rise higher if it has been in the background for too long. Even more if the app consumes too much memory. This is one of many reasons why all applications should aim for a low memory footprint.
Getting killed like this is commonly referred to as an Out of Memory Termination (OOM). But before this event actually happens, the OS will send a memory warning notification.
2. Memory Warnings.
Before killing an application, iOS sends a memory warning notification to it. When this happens, we need to do 2 things: To release as much memory as possible and to prepare for the worst.
At the end of the article, we will explore some ideas on what to do when receiving a memory warning.
3. OOM-Free Sessions.
When the app gets killed while running in the foreground, it means it crashed. It can happen, but iOS will kill background apps first. When an app gets killed while running in the background, that’s considered an OOM Session. Apps running in the background are the ones with the highest chances of getting jettisoned.
OOM-Free Session rate is a great metric to look after in Fabric’s Crashlytics or any other crash reporting tool. This will indicate if users are experiencing OOM terminations often or not.
A low OOM-Free rate means we probably should take a look at how the app is using memory. We want to keep this number high (namely, keep the number of OOM Terminations Low).
OOM Terminations can be quite frustrating for users in terms of UX. We open an application, send it to the background and when we open it again, it starts from zero. Even more frustrating if we lose progress in between sessions. In any case, we want to keep the OOM-Free sessions as high as possible. In order to do so, we need to aim for a low memory impact.
4. Image Rendering.
Image rendering can be quite expensive in terms of memory consumption. I will briefly explain the process so we understand why. The process can be split into 2 phases: Decoding and Rendering.
For each phase, there is a memory buffer involved. A data buffer is used by the decoding phase to create an image buffer, which is used by the rendering phase.
The image buffer created in the decoding phase is the one that will have a higher impact on memory footprint in the long term.
The Decoding Phase is the transformation of the contents of the compressed image data (data buffer) into information that can be interpreted by the display hardware (image buffer).
The image buffer contains the in-memory representation of the image used by the display hardware. Such information comprises the color and transparency for each pixel. The Rendering Phase is when the hardware consumes the image buffer and actually “paints” it on the display.
Let’s say we want to display a 12MP photo taken by an iPhone8 in a small frame of 350 pixels height, that could be a collection cell for example. The uncompressed representation of such photo will be loaded into memory in the image buffer.
The size of the buffer is calculated by multiplying the number of bytes per pixel (which comes from the color profile) by the width and height of the image. Therefore, image size and color profile are the 2 aspects that determine the size an image will occupy in memory.
What happens if we display a 12MP photo using UIImage(named:)? The photo is 3024 x 4032 pixels and with Color Space is RGB with DisplayP3 Color Profile. Such color profile takes 16 bits per pixel. Therefore, such a photo will take roughly 3024 x 4032 x 16 bits. That is 3024 x 4032 x 8 bytes ~= 93.02 MB.
Take a look at this screenshot taken from the profiler when loading such an image using UIImage(named:). It is clear that the report aligns with the predicted calculation.
It takes a staggering amount of 93MB of memory to display an image inside a small area of 350 pixels height. Rendering on such a small area should not be that expensive. This is becauseIImage(named:) creates a memory buffer for the full size and quality of the source image. This is not ideal in some situations.
It’s also important not to confuse image size in disk vs image size in memory. Disk/bundle footprint has to do with image compression format, like jpeg or png, and it does not correlate to memory usage when rendering the image. Again, this only depends on the image dimensions and the color profile.
So, what alternatives do we have?
Techniques for a lower memory footprint.
The following list of techniques focuses mostly on lowering the memory impact of displaying images.
Technique #1: Color Profile.
Color Profile. This impacts on the number of bits per pixel used to represent the image in memory. sRGB uses 8 bits, display p3 uses 16 bits. This could create a really big difference.
Take a look at this comparison from the profiler, the same image with different color profiles consumes a different amount of memory. DisplayP3 consumes 93MB and sRGB consumes roughly 46.5MB (half).
So, the images might be created with DisplayP3 profile, but sometimes it makes no sense to display such high-quality images. In those cases, we can convert the image color profile to sRGB and save half of the memory.
Technique #2: UIGraphicsRenderer from Image I/O.
An alternative toUIImage(named:) is to use aUIGraphicsRenderer. This is a very powerful tool that lets us draw geometric figures, colors, and even images in a context that is created for a specific size.
The image can be huge, but the area in which we want to render it can be quite small. UIGraphicsRenderer uses a memory buffer of the size of the frame that will be rendered, instead of the full image frame. This buffer contains per-pixel information of color and transparency.
How can we render images using UIGraphicsRenderer?
We need to load the image data from the disk and create a CoreGraphics image that the renderer context will draw. The context will use a rectangle of the appropriate size, consuming less memory. The scale allows us to increase the size of the rectangle and can be used as a trick to increase the quality of the image.
In the following extension, we can pass several URLs to draw more than one image at a time:
Creating another extension to actually render a single image is quite straightforward:
And finally, this is how to actually use the code above:
I’ve created a sample project with some comparisons and the full code for this and other techniques explained in this article. Feel free to check it out.
Let’s see how much memory we can gain from using this technique:
Bear in mind that this technique results in a significant decrease in the memory buffer size, but it also loses quality on the resulting on-screen image:
Clearly, the image on the right is a little blurry. It was rendered using a scale of 1. If we increase to a scale of 4 for example, the difference is almost unnoticeable but the memory gain is also huge.
Important: I haven’t found a way to get the path of an image that is bundled inside an Assets Catalog (if you know a way please let me know, I’d appreciate it 🙂). So, to use this technique with bundled assets, we need to place them outside the Assets Catalog.
This tool replaces the oldUIGraphicsBeginImageContextWithOptions which used to be the go-to tool from CoreGraphics. UIGraphicsRenderer recognizes automatically the color space needed to use and takes advantage of the Display P3 profile if necessary, whereas UIGraphicsBeginImageContextWithOptions uses sRGB at the most.
Also as we will see in incremental image loading, an UIGraphicsRenderer can be configured with different sources to achieve different things.
Technique #3: Use UIGraphicsRenderer for composite images.
Sometimes we need to render multiple pngs to display a resulting image. Like a map that needs several countries marked as visited, or the human body, with several parts marked in some color to indicate pain or activity. An option to implement this is to have a lot of pngs with transparency, so we don’t have to deal with the positioning of different pngs.
The problem with such an approach is that each image will occupy space in memory, on memory buffer per image. And each memory buffer is of the size of the original image. So, for every layer/image to render iOS needs the one memory buffer.
Instead, we can use UIGraphicsRenderer to render the 10 images in the same context. That will use only one memory buffer of the size of the frame to render the final image on. This can result in a huge win if we are dealing with multiple layers of transparent images to create a final image.
Technique #4: Load images incrementally using Image/IO.
Imagine that a user scrolls quickly through a collection view that contains a lot of images. Each image appears on the visible part of the collection and is quickly disposed of because of the scrolling speed.
Image I/O UIGraphicsRenderer can be used with a different type of source, an incremental one. An incremental source receives chunks of data and an image renderer can produce images as the new data is received.
So, we can create the image source, pass data as we receive it and render the resulting image. In the case of the collection of the example, we could use a timer to load the rest of the image for those cells that are still alive, avoiding the spikes caused while scrolling and having to render the full images.
I added code to load images incrementally in the GitHub Project. I’ve based some of it from ImageIOSwift from Dave Beck, which has a lot of functionalities, it’s really worth checking!
Technique #5: Downsampling with Core Graphics.
This technique is covered quite deeply in this amazing session from WWDC 2018. As we’ll see, it can dramatically decrease the amount of memory that image rendering consumes.
Downsampling is similar to using UIGraphicsRenderer. It uses a buffer of the size that’s necessary on the screen, instead of the dimensions of the source image. And also, we can use this technique if we have to display large images and we can compromise a little quality to use less memory.
We can use downsampling for local images or images loaded from HTTP Responses or any other source that produces NSData:
Find the full code in the sample project.
The following screenshot shows that the savings are huge! This is using a scale of 1.
A difference I’ve found with UIGraphicsRenderer is when increasing the scale, or the size of the frame, for some reason it was not linear. When multiplying it 4 times, it created an 11MB buffer. I haven’t done much testing, but it is still a huge saving compared to the original 93MB(Display P3) or 48MB(sRGB).
Technique #6: Download smaller images.
This is not much of a technique, but more of common sense and communication. Nonetheless, it will improve not only memory usage but also network bandwidth consumption.
If we are displaying small images, in a table or collection view, there might be no need for the large full detail image. We could use a smaller size image (in terms of dimensions, not format) which will cost less bandwidth to download and render faster consuming less memory.
Asking the backend team to create access to thumbnail versions of the full images could be a good option. There are tools to automatically convert large images to thumbnails when requesting them on downloads, like imgproxy.
Technique #7: Managing Caches.
Apps usually have several caches, like URL Caches, Image Caches, etc. Defining the right size for each cache and purging them at the right time, can cause significant memory improvements.
Purge Image Caches + HDD caching: We can purge image caches when the app goes to the background, for example. This way we will decrease the chances of OOM terminations because the footprint will be lower.
If we combine this with some sort of HDD caching, we can still save bandwidth and the app can perform well, while not occupying too much memory.
Cap the cache size: If we control the size of the caches, we can make them occupy less memory. For example, using Alamofire we can configure the max cache size like this:
Technique #8: Unload unused sections of the app.
Sometimes there is no need to have certain view controllers living behind the scenes, waiting to be used. Especially if they occupy too much memory. In those cases, we could unload them from memory until they appear again.
I’m talking about releasing references to heavy view controllers that are on a tab that is not visible, for example. We could load them on-demand again when it is necessary.
Technique #9: Get rid of memory leaks.
By definition, memory leaks increase the memory footprint of apps. I wrote an entire post about memory leaks in Swift. Leaks are dangerous and can generate crashes besides wasting memory.
I try to profile often and look for spikes in memory besides leaks using the Memory Gauge Tool from XCode and Instruments too.
What to do on memory warnings?
Do not purge NSDictionary based caches
Interestingly, do not purge image caches that are implemented with plain NSDictionary. Doing so will increase the type of memory called Compressed Memory. This will cause the app to use even more memory, increasing the chances of getting killed. This topic is covered in detail in this 2018 WWDC Session, check out minute 5:50.
NSCache based implementations should be purged though!
Release resources
Release as much memory as possible, to give the app a chance to keep running. Unload view controllers that are not visible, release images, model files, whatever you can think of.
Save progress and more.
We need to prepare for the worst-case scenario and provide users with a good experience even if iOS terminates the app.
Saving the progress to allow users to recover the state is a must, considering what is mentioned above. Automatic recovery is another good alternative that’s worth considering. It all depends on the application and the cost/benefits balance of implementing such features.
Conclusion
In an ecosystem where multiple apps are running at the same time and competing for scarce memory, having a low footprint is key. The less memory used, the lower the chances of getting killed and the better the app behaves with others. This impacts directly in the battery life, CPU, and most importantly in the UX.
We’ve seen how we can dramatically decrease the memory used by applications using some techniques. Mostly image rendering based techniques from ImageIO and CoreGraphics. With the help of profiling tools and very little code, we can achieve great improvements.
What do you think? Do you know other techniques to contribute to this list?
