Building a DocumentsProvider

One of the Android’s strengths has always been its intent system: rather than whitelist only specific apps your app works with, you can rely on common intents that define standard actions that apps can register to handle. One of these common actions is retrieving a specific type of file, often implemented with ACTION_GET_CONTENT.

Prior to KitKat, this meant building an Activity that has an ACTION_GET_CONTENT intent filter, writing a UI that allows selecting files managed by your app, and handle a myriad of flags like multi-select and local only. And then users had to learn each app’s UI. The wild west of file selection. So in KitKat, we introduced a new standard with the Storage Access Framework and the class that underlies the entire system: DocumentsProvider.

This allows users to have a single standard user interface to access files from any app — whether they are local files from the included local storage DocumentsProvider or from a custom DocumentsProvider you build.

What is a DocumentsProvider anyways?

A key distinction between the old system and the Storage Access Framework is that the UI is provided by the system, not directly by your app.

The system provides the UI for all DocumentsProviders

A DocumentsProvider then has a single focus: providing the information needed to populate that UI with the directories and files (collectively known as ‘documents’) managed by your app.

You might have guessed it, but a DocumentsProvider extends ContentProvider — one of the high level components available on Android that are particularly well suited for allowing other apps (or, in this case, the system) to read information from your app and provide access to files you own.

And just like any ContentProvider, that means your DocumentsProvider needs to be registered in your manifest:

<provider
    android:name="com.example.YourDocumentProvider"
    android:authorities="com.example.documents"
    android:exported="true"
    android:grantUriPermissions="true"
    android:permission="android.permission.MANAGE_DOCUMENTS">
  <intent-filter>
    <action
      android:name="android.content.action.DOCUMENTS_PROVIDER"/>
  </intent-filter>
</provider>

You’ll note the authorities attribute — this need to be a unique string you can think of as a prefix for all of the URIs that your DocumentsProvider builds. We’ll be referring to this in code as well, so it sometimes makes more sense to do some Gradle magic to ensure they always stay in sync:

// build.gradle
defaultConfig {
  def documentsAuthorityValue = applicationId + ".documents"
  
  // Now we can use ${documentsAuthority} in our Manifest
  manifestPlaceholders =
    [documentsAuthority: documentsAuthorityValue]
  // Now we can use BuildConfig.DOCUMENTS_AUTHORITY in our code
  buildConfigField "String",
    "DOCUMENTS_AUTHORITY",
    "\"${documentsAuthorityValue}\""
}

Now we can update our provider to use android:authorities=”${documentsAuthority}” and use BuildConfig.DOCUMENTS_AUTHORITY anywhere in our code to refer to the same constant.

Note: applicationId will be null in a library module so this technique only works if your DocumentsProvider is within an application module.

Thankfully, DocumentsProvider takes care of the high level ContentProvider APIs, giving you a very document specific API to implement.

It all starts at the root

And that document specific API starts with queryRoots(). A ‘root’ is the topmost entry that appears in the Documents UI and includes information such as a unique root ID, the root’s ‘display name’ (the user visible name), an icon, an optional summary, and most critically the document ID of the topmost directory for that root (which is what it’ll use to actually enumerate the rest of your content).

For most apps, this is pretty straightforward — you’d have one root for your app, but this doesn’t necessarily need to be the case. For example, the UsbDocProvider in the above diagram has a root for each connected USB drive (which in practice means most of the time it has no Roots at all and doesn’t appear in the list — as you’d expect). You could also consider the case where you support multiple accounts: you can and should have a separate root for each user account.

Working with Cursors in a DocumentsProvider

When you actually look at the full definition of queryRoots(), you’re met with two concepts right from the start: a Cursor and a projection. These are common terms when it comes to working with databases (and actually the basis behind a ContentProvider), but that doesn’t mean you need to know the details of databases to write a DocumentsProvider.

So just like a traditional database has a number of rows with each row made up of a number of columns, queryRoots() is expecting a Cursor that has a row for each root you want to return with each root having a number of columns representing the various bits of information. The projection that is passed in is an array of which columns are being requested or null if you get to pick what columns you want to return.

In the case of a document root, the valid columns are found in DocumentsContract.Root — this ‘contract’ is what the system and your DocumentsProvider need to agree on. In the Root’s case, there are a number of required columns:

• COLUMN_ROOT_ID — a unique String defining the root. As long as this is unique within your app, it can be anything you want
• COLUMN_ICON — a resource ID of an icon to display for the root. Ideally this should be something branded such that it is clear what app the root is associated with
• COLUMN_TITLE — the title of the root — this should be a user friendly name (keep in mind there’s a separate, optional COLUMN_SUMMARY for things like an account name)
• COLUMN_FLAGS — an integer representing what optional behavior your root supports such as whether it represents local only data, or if you support creating new files, sorting by recency, or searching. If you don’t support anything, this can just be 0
• COLUMN_DOCUMENT_ID — a String for the topmost directory of the root — this is how the Storage Access Framework is going to start exploring your root once someone selects it

So it makes sense to build a default root projection that includes at least these fields:

private final static String[] DEFAULT_ROOT_PROJECTION =
  new String[]{
    Root.COLUMN_ROOT_ID,
    Root.COLUMN_ICON, Root.COLUMN_TITLE,
    Root.COLUMN_FLAGS, Root.COLUMN_DOCUMENT_ID};

Now you can use a MatrixCursor to manually build a Cursor with the required columns:

MatrixCursor result = new MatrixCursor(projection != null ?
  projection : DEFAULT_ROOT_PROJECTION);

And for each root you want to add, call newRow():

MatrixCursor.RowBuilder row = result.newRow();
row.add(Root.COLUMN_ROOT_ID, rootId);
row.add(Root.COLUMN_ICON, R.mipmap.ic_launcher);
row.add(Root.COLUMN_TITLE,
  getContext().getString(R.string.app_name));
row.add(Root.COLUMN_FLAGS, Root.FLAG_LOCAL_ONLY |
  Root.FLAG_SUPPORTS_CREATE);
row.add(Root.COLUMN_DOCUMENT_ID, rootDocumentId);

Don’t worry about checking whether the projection includes each column you want to add — they’ll be ignored if they aren’t needed.

Dynamic Roots

If you’re doing anything more than a static set of roots, it is critical that the document UI stay in sync — the user shouldn’t have to see an already disconnected USB device or a signed out account. Thankfully, being built on a ContentProvider gives us a pre-built mechanism for notifying listeners of changes via notifyChange():

Uri rootsUri =
  DocumentsContract.buildRootsUri(BuildConfig.DOCUMENTS_AUTHORITY);
context.getContentResolver().notifyChange(rootsUri, null);

Onto the documents

Once the user sees and selects your root, you’ll want to actually return some documents from queryChildDocuments(). This may take the form as more directories (which would then be recursively explored by the user) or files at that level in the hierarchy.

Just like queryRoots(), this takes a projection and returns a Cursor. These function the same, but will use the columns defined in DocumentsContract.Document, which also has a number of required columns:

• COLUMN_DOCUMENT_ID — a unique String that identifies this document
• COLUMN_DISPLAY_NAME — the user visible name of the document
• COLUMN_MIME_TYPE — the MIME type of the document e.g., “image/png” or “application/pdf” — use MIME_TYPE_DIR to represent a directory
• COLUMN_FLAGS — an integer representing what optional behavior this specific document supports.
• COLUMN_SIZE — a long representing the size of the document in bytes (if you don’t know, you can add null)
• COLUMN_LAST_MODIFIED — the last modified date in milliseconds (if you don’t know, you can add null)

And the same MatrixCursor based approach works here as well.

Keep in mind that the COLUMN_DOCUMENT_ID must be a uniquely describe a single document, a document can have more than one parent — it is completely valid to have the same document appear in multiple places in your directory structure. For example, your DocumentsProvider could have one branch of its directories sort images by user defined tags and another by year — the same image might appear until multiple tags as well as under a year directory.

While queryChildDocuments() is the primary method that’ll be called when the user is exploring your DocumentsProvider, you must also implement queryDocument() — this method should return the exact same metadata about just a single document as you would have returned in queryChildDocuments(). (This is a good opportunity to dedup your code and have one set of code that builds a row for both queryChildDocuments() and queryDocument()).

Loading from the network

Of course, if you’re working with user’s files, one user will eventually have a directory of tens of thousands of files. If you’re loading the file metadata across the network, the user might be sitting there for quite some time. Instead of loading the entire set in one chunk, a DocumentsProvider allows you to specify EXTRA_LOADING to indicate that there’s more documents coming:

MatrixCursor result = new MatrixCursor(projection != null ?
  projection : DEFAULT_DOCUMENT_PROJECTION) {
    @Override
    public Bundle getExtras() {
      Bundle bundle = new Bundle();
      bundle.putBoolean(DocumentsContract.EXTRA_LOADING, true);
      return bundle;
    }
  };

Yes, you need to extend your Cursor class to override getExtras(). (There’s also EXTRA_INFO and EXTRA_ERROR which you may find useful in displaying information to the user using this same technique).

Then we can use the same notifyChange() based approach as used for dynamic roots, but here we need to set a specific notification Uri on our Cursor (as there are many Cursors when it comes to documents):

result.setNotificationUri(DocumentsContract.buildChildDocumentsUri(
  BuildConfig.DOCUMENTS_AUTHORITY, parentDocumentUri);
// When we’ve loaded our data
Uri updatedUri = DocumentsContract.buildChildDocumentsUri(
  BuildConfig.DOCUMENTS_AUTHORITY, parentDocumentUri);
getContentResolver().notifyChange(updatedUri, null);

Recents and Search

Two of the optional flags you can include when returning your root are FLAG_SUPPORTS_RECENTS and FLAG_SUPPORTS_SEARCH. These flags denote that users can get a list of recently modified documents and search the root, respectively. In both cases, you’ll use the same technique as queryChildDocuments().

Recent documents are returned via queryRecentDocuments() and should be limited to at most 64 documents in descending order of COLUMN_LAST_MODIFIED. Documents you return here will be combined with others under the system provided ‘Recent’ root, allowing users to get an overview of their most recent documents across all installed DocumentsProviders.

Note: Recent documents does not support notifyChange(), unlike queryChildDocuments() and querySearchDocuments() given that the results are combined across many DocumentProviders by default (too much shifting around if they were all updated at different times!).

Searching a root requires that you implement querySearchDocuments(). Here you’re given a specific query string and need to return the most relevant documents. While at a minimum this should attempt to match the COLUMN_DISPLAY_NAME (case insensitive), but it could certainly look at other metadata — tags on an file, OCR of images, etc. Just make sure it is actually returning relevant results!

Getting to the meat of a document: the bytes!

In most cases, the reason the user is going through this whole process to actually select and open a file. So it makes sense that the last method you must implement is openDocument() — how you actually provide the raw bytes of the document.

Here, you’re tasked to return a ParcelFileDescriptor — a bit more than your standard OutputStream, sure, but surprisingly a bit more flexible, supporting both reading and writing. If you already have a local file representing your document, this becomes a rather trivial method to write:

public ParcelFileDescriptor openDocument(final String documentId,
    final String mode,
    final CancellationSignal signal) throws FileNotFoundException {
  // Get a File from your documentId,
  // downloading the file if necessary
  File file = …;
  return ParcelFileDescriptor.open(file,
    ParcelFileDescriptor.parseMode(mode));
}

Note: if you’re syncing the file elsewhere and need to know when the file was closed, consider using the open() call that takes an OnCloseListener to know exactly when the other app is done writing to the file — be sure to check the IOException to know if the remote side actually succeeded or ran into an error.

Of course, if you have a more complicated structure or are streaming the file, you’ll want to look at createReliablePipe() or createReliableSocketPair() which allow you to create a pair of ParcelFileDescriptors where you’d return one of them and send data across the other via a AutoCloseOutputStream (for sending data) or AutoCloseInputStream (for receiving). These cases wouldn’t support the “rw” read+write state — that case assumes random access and a local file.

Note: if the CancellationSignal is not null, you should check its isCanceled() method occasionally to abandon long running operations.

Providing Thumbnails

By default, each document uses a default icon based on its mime type. This can be overridden by providing a custom icon by including the COLUMN_ICON, but for documents like images or videos (or even documents/PDFs), a thumbnail can make all the difference in letting the user figure out which is the correct document to select.

When you add FLAG_SUPPORTS_THUMBNAIL to a document, the system will call openDocumentThumbnail(), passing in a size hint — a suggested size for the thumbnail (as mentioned, the image should never be more than double the hinted size). Since these are going to be visible as part of the browsing process, caching these thumbnails (say, in getCacheDir()) is strongly recommended.

Note: you’ll find that the AssetFileDescriptor can be created from a ParcelFileDescriptor by using new AssetFileDescriptor(parcelFileDescriptor, 0, AssetFileDescriptor.UNKNOWN_LENGTH)

And if you have a whole directory of documents that support thumbnails, your users will probably appreciate it if you set FLAG_DIR_PREFERS_GRID on the parent directory to get larger thumbnails by default.

Virtual Files

While it makes sense that documents with MIME_TYPE_DIR aren’t openable (they are directories after all!), there’s another class of documents that aren’t actually directly openable — these are called virtual files. New to Android Nougat and API 24, a virtual file is denoted by the FLAG_VIRTUAL_DOCUMENT. These files won’t be selectable when apps include the CATEGORY_OPENABLE category in their intent and won’t ever call openDocument().

Why include them at all then? Well, the ACTION_VIEW intent sent when a user clicks on the file in the included file explorer (Settings->Storage->Explore on Nexus devices) will still work with these files, allowing the user to open the files within your own app.

Virtual files also particularly benefit from one of the other features added in API 24 — alternate file formats. This allows virtual files to have alternate openable file export formats (such as a PDF file for a cloud document).

Alternate File Formats

There’s an implicit connection between the COLUMN_MIME_TYPE you return and what you return in openDocument() — if you say you’re an “image/png”, you had better be delivering a PNG file. On API 24+ devices there’s an additional option though: allowing apps to access your document through alternate mime types. For example, you might be providing “image/svg+xml” files and want to also allow apps to use a fixed resolution “image/png” given the lack of native SVG parsing which would normally make an image/svg+xml file less useful.

Here, your DocumentsProvider can implement getDocumentStreamTypes() to return a full list of mime types that match the given mime type filter (e.g., “image/*” or “*/*”) that are supported by the given document id. Keep in mind that you should include your default mime type if it represents an openable mime type.

Then when clients use openTypedAssetFileDescriptor() with one of those mime types, that’ll trigger a call to openTypedDocument(), which is the mime type equivalent to openDocument().

Going beyond ACTION_GET_CONTENT

Much of what we’ve talked about is in building the best experience on KitKat devices for client apps using ACTION_GET_CONTENT and while clients can use DocumentsContract.isDocumentUri() to determine if they’re actually receiving a document Uri (a good way of conditionally using the more advanced functionality provided), by implementing a DocumentsProvider you’ll also allow clients to use ACTION_OPEN_DOCUMENT, which ensures that all Uris returned are document Uris as well as allow persistent document access). There are two other actions which your DocumentsProvider can optionally handle: ACTION_OPEN_DOCUMENT_TREE and ACTION_CREATE_DOCUMENT.

Note: If your app contains a DocumentsProvider and also persists URIs returned from ACTION_OPEN_DOCUMENT, ACTION_OPEN_DOCUMENT_TREE, or ACTION_CREATE_DOCUMENT, be aware that you won’t be able to persist access to your own URIs via takePersistableUriPermission() — despite it failing with a SecurityException, you’ll always have access to URIs from your own app. You can add the boolean EXTRA_EXCLUDE_SELF to your Intents if you want to hide your own DocumentsProvider(s) on API 23+ devices for any of these actions.

ACTION_OPEN_DOCUMENT_TREE

While ACTION_GET_CONTENT and ACTION_OPEN_DOCUMENT focus on providing access to one or more individual documents, ACTION_OPEN_DOCUMENT_TREE was added in API 21 to allow a user to select an entire directory, giving the other app persistent access to the entire directory.

Supporting ACTION_OPEN_DOCUMENT_TREE involves adding FLAG_SUPPORTS_IS_CHILD to your root and implementing isChildDocument(). This allows the framework to confirm that the given document ID is part of a certain document tree: remember a document can be in multiple places in your hierarchy so this is checking ‘downward’ as it were from parent to potential descendant (child, grandchild, etc).

Ideally, this request shouldn’t rely on the network as it can be called frequently and in rapid succession so if you want to support this use case, make sure you can handle this request locally.

ACTION_CREATE_DOCUMENT

If ACTION_OPEN_DOCUMENT is the ‘open file’ of a traditional operating system, ACTION_CREATE_DOCUMENT is the ‘save file’ dialog and allows clients to create entirely new documents within any root that has FLAG_SUPPORTS_CREATE in directories that have FLAG_DIR_SUPPORTS_CREATE (think of the root flag as a flag on whether your root should appear at all in the UI for ACTION_CREATE_DOCUMENT, rather than a blanket grant to create documents anywhere).

When the user selects a directory to place the new document, you’ll receive a call to createDocument() with the parent document ID of the selected directory, the mime type specified by the client app, and a display name (ideally, what you should set as the COLUMN_DISPLAY_NAME, but you can certainly edit it, add an extension, etc. if needed). All you have to do is generate a new COLUMN_DOCUMENT_ID which the client app can then use to call openDocument() to actually write the contents of the document.

Document Management

While the core experience will always be driven by reading, writing, and creating documents, a traditional file manager has quite a few more features and many of them are supported when building your DocumentsProvider, allowing the system UI to offer additional functionality for those wanting to browse or manage your documents.

Each functionality has a flag you need to add to document’s COLUMN_FLAGS and an associated method you need to implement to perform the operation:

• FLAG_SUPPORTS_DELETE (API 19) to delete the document entirely. Implement deleteDocument().
• FLAG_SUPPORTS_RENAME (API 21) to rename the document (changing its COLUMN_DISPLAY_NAME and optionally its COLUMN_DOCUMENT_ID). Implement renameDocument().
• FLAG_SUPPORTS_COPY (API 24) to create a copy of the document under a new parent directory within your DocumentsProvider. Implement copyDocument().
• FLAG_SUPPORTS_MOVE (API 24) to move a document from an existing parent directory to a new parent directory within your DocumentsProvider. Implement moveDocument().
• FLAG_SUPPORTS_REMOVE (API 24) to remove a document from its parent directory. Implement removeDocument().

Keep in mind the subtle difference between delete and remove: remove is focused more on the relationship between a document and its parent directory. In the case where you only have one parent of the document, a single remove would orphan the document and you’d usually want to trigger the same code as delete (since there’d be no way to navigate to that document any more). But in the case of multiple parent directories for a single document, delete would affect every parent while remove would be a local operation only affecting a single parent directory.

When it comes to operations that can remove existing document IDs (such as delete, move, and removing from the last parent), make sure you call revokeDocumentPermission() — this is what tells the system that the document ID is no longer valid and access should be revoked from all other apps.

DocumentsProvider: storage for the modern era

Local storage is certainly still incredibly important and that’s why the system provides DocumentsProviders for internal storage, SD cards, and USB attached storage devices. With the Storage Access Framework, your app and the data you store on behalf of the user is now just as accessible as if it was right there on the device with them.

#BuildBetterApps

Follow the Android Development Patterns Collection for more!