Fixing disappearing drag and drop items on iOS

Since the release of Drag and Drop in iOS 15, moving around photos, files and text from app to app or from one part of an app to another has been a breeze. How to implement this feature has already been well documented in Apple’s official documentation as well as several well written Medium articles, which I will leave a reference to at the end of this article. If you are not familiar with the basics of drag and drop, I suggest reading those articles/documentation before moving on.

This article is instead going to focus on an issue we faced at Ancestry when implementing drag and drop and which others may come across.

Here is the scenario:

We implemented drag and drop in a way that allows users to drag many photos into the in-app gallery. This gallery uploads photos 3 at a time (if three or more have been selected), asynchronously, while holding onto the rest of them while they wait their turn.

What we found was that when dragging more than 3 photos, only 3 photos would ever be uploaded, and the rest would seemingly be forgotten. If we changed the asynchronous upload limit to another number, such as 4 or 5, then we would see that number be uploaded, but never more.

We scratched our heads and tried various things before coming to the realization that, when receiving NSItemProviders from drag and drop, it’s possible to lose access to them while asynchronous code is running.

When the function to upload all our images runs, it grabs the first three item providers and begins uploading them, but somewhere along the way, we lose access to the other item providers.

How to get around this? It turns out the simple answer is to convert the item providers you get from drag and drop into images (or whatever form you are expecting) and then back into item providers to use wherever you need.

Let’s break this down and see what this function is actually doing

public static func convertItemProviders(dragItems: [UIDragItem], completion: @escaping((itemProviders:[NSItemProvider], failedToConvert:Bool))->Void) {

This function takes in an array of UIDragItems and a completion which contains a list of NSItemProviders and a bool, which tells us if any of the item providers were not able to be converted for some reason.

Here we are setting up some variables that we will need. Images will be an array of tuples that contain the images we get from the dragItems item providers, as well as their index in the list. The group will represent our dispatch group and the failedToConvertAnItem will let us know if one of our conversions failed. Finally, we take our dragItems and convert them into itemProviders and filter out any that cannot be images.

var images = [(image:UIImage, index:Int)]()
let group = DispatchGroup()
var failedToConvertAnItem = false
// Get item providers given to us by drag and drop
let itemProviders = dragItems.map({$0.itemProvider}).filter({$0.canLoadObject(ofClass: UIImage.self)})

Next, we enter a dispatch group and take our item providers and their associated index and we load the data from it. If we encounter an error, we set our failedToConvertAnItem bool to true. If we get data, and we can make an image from it, then we append that image and the index to our array of image and index tuples.

for (index, itemProvider) in itemProviders.enumerated() {
    group.enter()
    itemProvider.loadDataRepresentation(forTypeIdentifier:  UTType.image.identifier) { data, error in
        // If we ever fail to convert an item, we want to alert the  user that one or more may have failed.
        if let _ = error {
            failedToConvertAnItem = true
        }
        if let data = data, let image = UIImage(data: data){
            images.append((image, index))
        }
        group.leave()
    }
}

When the for loop is finished, and we have finished with all our dispatch groups, we sort our image/index array of tuples and map it to an array of NSItemProviders. We will not lose access to this array of NSItemProviders during asynchronous code. We then call our completion with our array and our bool.

group.notify(queue: .main) {
    // Take the images, sort them and turn them into item providers
    let newItemProviders = images.sorted(by: { image1, image2 in
        image1.index < image2.index
    }).map({NSItemProvider(object: $0.image)})
    completion((newItemProviders, failedToConvertAnItem))
}

You can use this function anywhere you might use NSItemProviders in asynchronous code, for example, you might call some function right after receiving the NSItemProviders from the delegate method in a collection view:

func collectionView(_ collectionView: UICollectionView, performDropWith coordinator: UICollectionViewDropCoordinator) {
    // Convert item providers from drag and drop into stable ones made from UIImages
    DragDropItemProviderConverter.convertItemProviders(dragItems: coordinator.session.items) { result in
    if result.failedToConvert {
        // handle a failed converison 
    }
    // Use stable item providers here
    self.uploadImages(with: result.itemProviders)
    }
}

There you have it! Once you turn the NSItemProviders into stable ones, you will no longer lose access to them while asynchronous code is running. See a way this code could be improved? Want to take a challenge and convert this function into async/await code? Let us know in the comments!

Medium article on basics of drag and drop

Apple’s documentation on drag and drop

If you’re interested in joining Ancestry, we’re hiring! Feel free to check out our careers page for more info.