Browse 1,000 photos in 10 seconds in an iOS app

I never have time to look back the lots of pictures…

With this in mind, I created an iOS app called “SOMATO” that allows us to browse 100 photos in a second at maximum.

If you have 1,000 photos, you can view them in 10 seconds.

Well, to be honest, the 100 images/second speed is more for impact than anything else, but surprisingly, even when the speed is around 20 images/second, you can still feel nostalgic and soak in the memories. It’s free, so if you’re interested, please download it and give it a try.

‎High Speed Photo Album SOMATO

(Available on iPhone, iPad, and M1 Mac)

This article describes the implementation of this application.

Overview

In fact, Somato is not particularly technically advanced.

To put it bluntly, it just acquires images with Photos (PhotoKit) and displays them with UICollectionView.

The first prototype took less than an hour to build. I didn’t have to do anything special, just a straightforward implementation, and I was like, “Hey, it’s displaying in less than 1/100th of a second…! That’s what I thought. (When I made an app with a similar concept about 10 years ago, it didn’t work like this due to the limitations of device performance.

But of course, it was not so easy when it came to making a proper product…

Memory Issues

In the first prototype I made, I loaded all the 500 images I had in my iPhone as UIImage first (before starting to display them).

Since the size of the images should be large enough to be displayed in three rows (375x375 on an iPhone 12 mini), the memory usage was still not significant.

However, this is a different story when you have more than 10,000 images on your iPhone. In fact, the designer I worked with had 16,000 photos and videos on his iPhone. No matter how small the size, 10,000 images cannot be stored in memory as UIImage.

Furthermore, a single row display would require higher resolution image data.

The concept of the app is to be able to look back at a large number of photos at high speed, so I can’t just set a limit of “up to 1,000 albums”. 10,000 or 1,000,000 photos need to be able to be displayed at high speed without eating up memory.

Multi-threaded problem

Due to the above limitation of memory usage, it is necessary to cache a certain number of images in memory and display them, while discarding the old cache and reading them in the background thread at a pace of 1/100th of a second.

So, in addition to the main thread for drawing the screen, multi-threaded processing is required: a thread for managing the memory cache, a thread for retrieving images from local or iCloud, and so on.

It is also important to note that the cache should be in a Serial Queue to unify access, and the image acquisition should be in a Concurrent Queue for parallel processing.

The problem of video frame extraction processing

Since the concept is to “browse a large number of photos at high speed”, we also want to look back at videos at high speed, so Somato extracts a frame from the video every n seconds and plays it at high speed just like a single photo. (This is quite emotive.)

This video processing is even more difficult, because the process of extracting frames involves video decoding, so no matter how much you read ahead while displaying 1/100th of a second, you will not be able to finish in time.

Therefore, it is necessary to cache the extracted frames, and then load the necessary portions into memory while discarding them.

The Problem with iCloud

iCloud was even more egregious. There were two main problems

1. Problems with the loading speed from iCloud
2. PhotoKit’s Internal Error problem

The first was a problem with downloading over the network.

For the first one, when downloading over the network, it naturally takes much longer than loading from the local network, so when doing PHImageManager.default().requestImage, first getting low-resolution thumbnail-level images with:

let options = PHImageRequestOptions()
options.deliveryMode = .fastFormat

then getting the necessary size images from iCloud and caching the files. These are done in a background thread while playing at 100 images/second.

However, the more complicated and inevitable problem for third-party developers is #2. This is caused by the iCloud sync status of your system. If you go to the Settings app, turn off Photos > iCloud Photos, wait 5 minutes (after the data is deleted), turn it back on, and then go to the standard Photos app, you will see that iCloud shared albums and other photos do not appear.

Immediately after enabling iCloud sync, or when you haven’t viewed the iCloud album (photos) in question for a long time in the standard Photos app, the OS hasn’t even been able to acquire thumbnail resolution images yet, and if you try to access the photos in the Photos framework at this time, it will cause the Internal Error.

[PhotoKit] Error: Unknown internal error

Conclusion

I wrote about the implementation of a viewer application that displays photos at high speed.

It was “easy to make” with about 500 photos, but when I started to consider about 16,000 photos, video support, iCloud support, etc., it was unexpectedly difficult.

There may not be many cases where you want to display images at 1/100th of a second, but

• Fetching images with Photos at high speed
• To save memory (not proportional to the number of images)
• Extracting frames from a video at high speed

I think this kind of implementation itself is actually rather useful in general.