Using the new list API in Cloud Storage for Firebase

Doug Stevenson
Aug 26 · 6 min read

Cloud Storage for Firebase recently launched support for listing files in buckets and folders for iOS, web, and Android clients. This was a hotly requested feature, because without it, developers wishing to upload many files to Cloud Storage generally had to make a database record of every object, so they could be reliably located and worked with later. While I think associating each file with a database record is still a good idea (more on that later), there are times when it’s very convenient to be able to get a list of files without having to first query a database.

Imagine you allow your users to upload audio recordings to Cloud Storage using a naming structure like this:

+ users/
|-+ {uid}/
| | — avatar.jpg
| |-+ audio/
| | | — {audioId}.mp3

{uid} and {audioId} are variable placeholders; the uid is a user’s ID as assigned by Firebase Authentication, and the audioId is some random unique identifier. avatar.jpg is a special image used for the user’s profile picture.

One thing to keep in mind first is that Cloud Storage doesn’t actually have “folders” like you’re accustomed to in a computer filesystem. It just has names with paths that look like UNIX folders. So, “users” and “audio” here aren’t actually a folders. What actually exists here are just objects with paths that look like this:

users/xyz/avatar.jpg
users/xyz/audio/foo.mp3
users/xyz/audio/bar.mp3

You can see that “users”, “xyz”, and “audio” are not individual objects, and they can’t be created or deleted using the Cloud Storage for Firebase SDKs. They’re just path components that are used to organize other objects, making it easier to think about the contents of your storage bucket, for the purpose of organization and navigation. However, sometimes we still refer to them as “folders” (especially in the Firebase console) to make it clear that they’re being used as organizers.

OK, with that settled, let’s say you want to delete the avatar image for a user. That’s easy, as long as you know how to build the path to that image. I’m using Kotlin on Android for my client code samples in this post. Code for other platforms will be similar.

val storage = FirebaseStorage.getInstance()
val uid = FirebaseAuth.getInstance().currentUser?.uid!!
val userAvatarRef = storage.reference
.child("users")
.child(uid)
.child("avatar.jpg")
userAvatarRef.delete()
.addOnCompleteListener { result ->
// Check result for errors
}

You would have probably already written similar code to create and update the avatar image in the first place. Pretty standard stuff.

Now, instead let’s say you want to delete all of a user’s audio files in organized under “audio” as shown above. You can try it like this, similar to the previous example:

val userAudioRef = storage.reference
.child("users")
.child(uid)
.child("audio")
userAudioRef.delete()
.addOnCompleteListener { result ->
// This actually will fail! Don’t do it like this.
}

But that fails with an error saying “Permission denied”. Why? Well, as I mentioned before, folders don’t really exist like fully pathed objects in Cloud Storage. But, what you can do here instead is use the new list API to list all the objects under that path. Or, to be more accurate, list the objects with that specific prefix. The word “prefix” is important here, as what we are actually going to do here is ask Cloud Storage to produce a list of all objects that begin with a common prefix string. Once we’ve listed those objects, then we can iterate and delete them individually, like this:

val userAudioRef = storage.reference
.child("users")
.child(uid)
.child(“audio”)
val listAllTask: Task<ListResult> = userAudioRef.listAll()
listAllTask.addOnCompleteListener { result ->
// should check for errors here first
val items: List<StorageReference> = result.items
for (item in items) {
item.delete() // returns another Task
}
}

Note that listAll() yields a list of StorageReference objects that point to the individual objects that exist at the given path prefix. The list operation is shallow, and doesn’t include files organized in deeper path components. The objects are organizationally siblings to each other.

If there are going to be a lot of objects in the list, you might have a problem. You could run out of memory storing all those references. Or you might not want to wait for the entire list to be generated at once. Instead, if you want to page through the results, you can call list() instead of listAll(). Check the documentation for an example of that.

And don’t forget about security rules

We’re not quite done yet! There’s a bit more to know. Anyone who has an app that directly reads and writes objects in Cloud Storage should be using Firebase security rules to protect who can perform what operations on objects stored in different paths. In the past, you probably just indicated “read” permission for download access. Before the list API was available, for the file example objects given above, the rules to allow anyone to download only a user’s avatar and audio files (and no other objects organized under the UID), might have looked like this:

match /users/{userId}/avatar.jpg {
allow read;
}
match /users/{userId}/audio/{audioId} {
allow read;
}

Now that the list API is available, you can control list permission separately from download permission. With Firebase security rules, read permission breaks down into get and list permission. The get method allows someone to download a single object (and access its metadata), and list allows someone to list objects (but not download them). Granting read permission is the same as granting both get and list, so any wildcarded object granted read access can also be listed with its siblings. However, this might not be what you want.

Here’s a specific case where you’ll need to take advantage of this split. Let’s say you want a user to be able to control download access to some, but not all, of their individual audio files. However, you don’t want anyone to be able to list the entire contents of their files, as that would reveal too much information. To make this work, let’s say that the user needs to indicate which objects should be publicly accessible by adding a piece of metadata to the object. Security rules can use this metadata to check if anyone should have access:

rules_version = "2";
service firebase.storage {
match /b/{bucket}/o {
match /users/{userId}/audio/{audioId} {
allow get: if request.auth.uid == userId ||
request.resource.metadata.public == “true”;
allow list, write: if request.auth.uid == userId;
}
}
}

First, note the very first line rules_version = "2";. This enables the use of the granular list permission used later. Without this line, rules will default to version 1, and you’ll get an error when publishing rules that use the list permission.

The above match on the path pattern /users/{userId}/audio/{audioId} is indicating that matched audio objects:

  1. Can be downloaded by the owner, based on their UID.
  2. Can be downloaded by anyone, if the custom metadata “public” contains the string value “true”.
  3. Can be written by the owner.
  4. Can be listed by the owner.

Also note that the rule ends with the wildcard {audioId}. This is required for the list API to work. The following rule will not enable the list API, because it won’t match the full paths of objects organized under audio:

match /users/{userId}/audio {
allow list; // This does nothing
}

One last thing to note is that read permission only implies list permission with Cloud Storage security rules version 2. So, with version 1, read permission only provides get access. But with version 2, read provides both get and list. This also means that when you upgrade your rules to version 2, your existing read permissions also start providing list permission. So, you should review your rules to make sure this is really what you want to allow.

To learn more about Firebase security rules for Cloud Storage, take a look at the documentation. They’re an important part of making sure the data you put in Cloud Storage is properly controlled.

Tutorials, deep-dives, and random musings from Firebase developers all around the world. Views expressed are those of the authors and don’t necessarily reflect those of Firebase or its parent companies.

Doug Stevenson

Written by

Google Developer Advocate with the Firebase team

Firebase Developers

Tutorials, deep-dives, and random musings from Firebase developers all around the world. Views expressed are those of the authors and don’t necessarily reflect those of Firebase or its parent companies.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade