The Customisation High Road: Dragging Items with Cocoa

An article about implementing drag and drop in a native macOS application using Apple’s Cocoa framework with custom images for dragged items. Discover more articles and showcases on augustfreytag.com.

Outset

For this example, we would like to set up a view serving some items that can be selected, dragged, and moved to some destination with the mouse or trackpad. In a practical case, they can also be dropped somewhere through a drag operation, though this is not the primary goal of this example. The following graphic shows a simple collection view with one section header and selectable items within. When starting off, no further interaction is available for the selected items.

Our concern begins with a view conforming to NSDraggingSource and would end with another one conforming to NSDraggingDestination. While the former must be used and implemented manually, the latter is adopted by default by NSView and the various others subclassing it.

While dragging and dropping can be theoretically implemented by any view, we’re going to be using an NSCollectionView because (1) it already has mechanics to list items and (2) enforces a useful, common pattern, that could itself be applied in less prepaved areas. In short, an NSCollectionView makes for an easier entry with its built-in functionality compared to a faceless view. Collection views remain a commonly used view class for visualising indefinite amounts of content, supporting sectioning, lazy loading and lazy redrawing while also — conveniently — allowing newcomers to get to know the delegate and data source design patterns first hand.

To summarise what is on schedule: A user selects one or more items and starts to drag them. At this point, codable representations are written to one or more pasteboards that are in turn available to the dragging destination later on. What is not clear from Apple’s official documentation are the different ways a drag can be started, how the images are created that follow the mouse (called either dragging frames or dragging images), and how the items and the dragging image can be accessed, changed, or even animated once the session starts.

Drawing

Drawing bitmap data directly to the active graphics context in general has many advantages, particularly performance improvements and customisation options down to each displayed dot. Visual elements can be rendered directly to the already focused graphics context in a view’s draw(_:) method. The spotlight for the moment is on preparing images for a dragging operation — the bitmap representation of items while they are being dragged.

Drawing this way involves care and a bit of conversion between the Cocoa type NSImage and the core graphics type CGImage, both built for different purposes despite seeming interchangeable at first. In a similar notion, we should know about NSGraphicsContext and the lower-level core graphics version CGContext, both of which can be used for drawing.

While this article will only cover the superficialities of drawing content for dragging items when they are created, the ability to freely provide a bounding rect and image contents allows for extensive customisation in positioning, scaling, transformation and using different bitmap contents.

Item Visualisation

When an item in the collection view is dragged, a phantom copy of that item is held by the mouse cursor and gets dragged to its intended destination until it is dropped. As with all other visual components of the application, the phantom is, in essence, a bitmap that was put together when the drag was initiated.

Built-In Single Image

While the Cocoa collection view in particular can already create a phantom image internally (called “dragging frame” or “dragging image”), there already is an open method to supply a custom one instead — our first entry point — belonging to the collection view’s delegate, a class conforming to NSCollectionViewDelegate.

The method is called when a dragging session starts and must return a single, non-optional NSImage that is supposed to represent all dragged items. The approach generally only makes sense when allowsMultipleSelection is disabled as only one image can be returned. If this is intended, any method to acquire a custom rendered image can be used to visualise the dragged contents. In this example, we're always going to be using the single selected collection view item view as a template, retrieved from the already available indexPaths supplied in the delegate's collectionView:didSelectItemsAtIndexPaths:.

The following code gathers the view owned by the selected NSCollectionViewItem that is returned from the collection view. If there were multiple items, they would need to be sorted as index paths are provided in a Set (that inherently does not include an order for its elements) and we want to draw our items in the order they appear in the collection view. Because items are subclasses of NSViewController, their associated view instance can be accessed and used as such.

From each view, an image can be created to represent it as it is drawn in the application. The most straightforward approach might be to use display caching functionaly provided with every NSView instance, as follows.

The resulting image representation can then be easily drawn as an NSImage instance, using one of its initialisers with a trailing drawing handler, init(size:flipped:drawingHandler:). The drawing procedure is straightforward and can reuse the view's bounds and dimensions.

Custom Images (In Bulk)

The behaviour included with NSCollectionView has the issue of only ever inquiring a single dragging item in the created session. A single item means also only using a single dragging image — this is somewhat lacking when dragging more than a single item as it can not be as easily represented in a single image than multiple images. Usually, when selecting and dragging a selection of items, each item might represent a standalone unit of data that is moved independently from items in the selection, the most common example being files. When dragging a selection of files from one directory to another, there should be one dragging item for each file added to the pasteboard, rather than one for all files.

It seems, NSCollectionView does not allow for a "multiple dragging items" approach when using its NSCollectionViewDelegate, though a manual reproduction is doable and explored below.

Inciting Event

The most appropriate event to hook into in this case — as its name would imply — is the event callback function mouseDragged, included in all classes based off NSView.

Once the mouse was dragged, we can create a dragging event ourselves, carrying the items we would like to transport. Because data is transported in pasteboards, the reader should keep in mind that the data to be dragged and dropped must be codable, i.e. conform to Codable or be encoded by other means.

Dragging Items & Data

As we have established, the dragging session carries dragging items. The items dragged in our case are always the same as the items selected. They are accessible via collectionView.selectionIndexPaths from the collection view. As a warning: A selection may not be immediately available once the drag starts if the "mouse down" was not yet registered, like when the first mouse dragged event may not have caused a click. A delay or drag delta can solve this issue if needed, examined and explained later.

To follow best practices, we won’t be creating items and the dragging session within the collection view itself but in its responsible delegate instead. Essentially, this functionality could be kept anywhere within the application and its exact placement is left up to the reader, though a delegate or other controller should be most suited for the task. The handling method might have the following form.

Each NSDraggingItem returned from here will contain the following:

• A pasteboard writer (instance conforming to NSPasteboardReading and NSPasteboardWriting, for example an NSPasteboardItem with a prepared, encoded value for a specific type)
• A dragging frame (a bitmap representation of the dragged contents)

To clarify these two options, the reader may imagine the pasteboard writer as the data being dragged and the dragging frame the data the user sees being dragged. Both of these should become obvious through the example implementations below.

While many different ways of supplying data to a dragging item exist, depending on the use case, at the time the drag occurs, the data is most likely fully available and can be encoded in full — thereby, a carrier pasteboard item can be created that fulfills the responsibility of a pasteboard writer to the dragging item (read back as “codable data > pasteboard item > pasteboard writer > dragging item”). It sounds like an overcomplicated step at first glance, though it can be realised with the following steps.

The alternative to the outlined approach is setting up a NSPasteboardItemDataProvider that will supply data on demand for each item. A pasteboard item can have a predefine data provider by using its setDataProvider method and providing a reference. For this example, we'll proceed with directly encoding data into the item by using setData as written in the snippet above.

The dragging items are returned to the function responsible to create a new dragging session; in our own example, the collection view itself in its mouseDragged function. The delegate has prepared a set of dragging items, the collection view itself will take over to create the dragging session. The session then receives the items and the original event that triggered the procedure.

Dragging Session

The session is started from the invoking view. The source specified here points to the collection view instance that noticed the dragging event. Some adjustments to the session's behaviour can be set, though these are optional.

At this point, the session has been initiated and will remain active until the user stops dragging the mouse. As a recap: We prepared items to represent the data to be dragged, encoded the data itself within pasteboard items and created dragging items for a new dragging session to begin on the responsible view, our collection view.

Because we are now supplying the dragging session with multiple items instead of just one, Cocoa takes care of the dragging formation, letting the items arrange in a list or even flock together automatically once the drag was started and moved. The formation can be changed to better represent how the dragged items are going to be used in any given context.

Supplementals

There are a few alternate routes that could be discussed. The approach described in detail before follows a very specific goal: one dragging item per source item and custom bitmap representations per item with arrangement of individual dragged items being handled by Cocoa itself. This is the behaviour observable when using the macOS Finder, selecting files and initiating a drag — the items are all visualised as individual images and flock together once in flight.

Alternative Single-Item Operation

The alternative approach using the built-in functionality of NSCollectionView can be used if the dragging image does not need to be customised and a single item suffices for the operation (like when dragging multiple items is disabled). The collection view only requires a single method for dragging source support to be enabled.

This delegate method fulfills two purposes in one, (1) write all items at the supplied indexPaths to the available pasteboard and (2) confirm a successful operation by returning true from the function.

iOS Equivalent

Interesting to note, the Cocoa Touch equivalent UICollectionView has more graceful implementation of drag and drop, even using separate protocols for dragging and dropping respectively. From a personal point of view, seeing that iOS has a more thought out and covering implementation available than macOS is as unsurprising as watching Xcode struggle with autocomplete. Nevertheless, the delegate method we'be using to fill a created dragging session is:

While only a single index path is supplied as parameter for reasons unknown, the function does allow returning multiple dragging items. This method is a strange inverted version of its regular Cocoa sibling, using sessions in the same way but providing a single index path instead of a set and returning multiple items instead of just one.