Introduction

So I have been playing with Firebase with a couple of prototypes and projects for quite some time. I have used React JS for a conference websitethat I co-managed and Polymer for a couple of prototypes that I am currently working on. After coming from using MongoDB and MySQL/PostgreSQL, information structure, storage and retrieval is very different in Firebase than with the two.

There are a lot of sites that explain how to structure data for easy storage and retrieval. The Firebase docs is the best way to start understanding how to structure your data (the old version also has some clues on anti-patterns that you need to avoid). Airpair has a good tutorial on translating tables and relationships into good data structure that is usable in Firebase.

This tutorial is taking what I’ve learned from both above, my experiences in structuring data, and the months of understanding how Firebase works.

To make it easy to understand, let’s create a hypothetical blog app. Your data objects might be like these:

Data structure for the hypothetical blog

Let’s add a few more details:

  • the User has a role: “admin”, “editor”, “contributor”, or “member”
  • a “member” can only create / edit / delete their Comment.
  • a “contributor” can do any functionalities the “member” can, and also create / edit / delete their own Article.
  • an “editor” can do any functionalities the “member” and “contributor” can, and can set any Article to be published and be seen by all. They can also give promote any “member” into a “contributor” as well.
  • an “admin” can do all of the above, and also can promote any “member”, “contributor”, or “editor” into an “editor” or “admin”.
  • An Article can only have one author (as of the moment, more on this later), but has many Comments.
  • An Article will only be viewed by all if it is published (published == true), or will only be viewed by the “admin”, “editor” or the owner if it is not published.
  • An Article can be found using the slug_name, which is the “slugified” title or the Article plus random characters for uniqueness.

Now given that, anyone who thinks Firebase is just another NoSQL would think of building it this way:

{
"user": {
"user1": {
"uid": "user1",
"firstname": "Jack",
"lastname": "Mayer",
"email": "jack_mayer@email.com",
"date_joined": 1333065600000
"role": "admin"
},
"user2": {
"uid": "user2",
"firstname": "Stef",
"lastname": "Gomez",
"email": "stef_gomez@email.com",
"date_joined": 1333065600000
"role": "contributor"
},
"user3": {...},
...
},
"article": {
"article1": {
"article_id": "article1",
"slug_name": "this-is-an-article-abcdef",
"uid": "user1",
"title": "This is an Article",
"body": "This is the body of the article."
"published": 1,
"date_edited": 1333065600000,
"date_created": 1333065600000
},
"article2": {...},
...
},
"comment": {
"comment1": {
"comment_id": "comment1",
"uid": "user2",
"article_id": "article1",
"body": "this is a comment",
"date_edited": 1333065600000,
"date_created": 1333065600000
},
"comment2": {...},
...
}
}

Looking above, it seems logical at first. But it breaks when you are going to retrieve things using Firebase, especially when you are now using the Firebase rules.

Firebase Retrieval and Rules

Now let’s say you want to list all Articles that are published but not show the Articles that are not published.

If this were in MongoDB, you can use this (I am going to use mongoose js for this):

var Article = mongoose.model('Article', yourSchema);  Article.find({ published: true }, function (err, articles) {
// list all articles here
})

And that’s done. But in Firebase it becomes different.

var articles = []firebase.database().ref('/article').orderByChild('published')
.equalTo(1).on('child_added', function(data) {
articles.push({
id: data.key,
data: data.val()
})
})

This is ok. This will show all Articles with published == 1 (or true, apparently you can’t use boolean values in equalTo). (See how it works here)

But if you removed equalTo, it will show all Articles even if published == 0. (See the jsfiddle here)

Adding a check on what to show in Firebase rules doesn’t work and renders the whole Article list not readable (see it not working here).

// Firebase Rules snippet"article": {
"$article": {
".read": "data.child('published').val() == 1"
}
}

And adding “.read” : true on article will allow rendering of all Articles regardless of the value of “.read” in each “$article” (You can see the whole reasoning here, See it not working as it should here).

// Firebase Rules snippet"article": {
".read": true,
"$article": {
".read": "data.child('published').val() == 1"
}
}

Note: In the JS fiddles above, I have created copies of the article list in article_with_rules_1 and article_with_rules_2. Their Firebase rules are set below:

A New List Structure

To remedy the problem above, we can change the structure a little bit. Instead of this:

{
...
"article": {
"article1": {
"article_id": "article1",
"slug_name": "this-is-an-article-abcdef",
"uid": "user1",
"title": "This is an Article",
"body": "This is the body of the article."
"published": 1,
"date_edited": 1333065600000,
"date_created": 1333065600000
},
"article2": {...},
...
},
...
}

I suggest we do this:

{
"article_group": {
"article": {
"article1": {
"article_id": "article1",
"slug_name": "this-is-an-article-abcdef",
"uid": "user1",
"title": "This is an Article",
"body": "This is the body of the article."
"date_edited": 1333065600000,
"date_created": 1333065600000
},
"article2": {...},
...
},
"article_list": {
"article1": {
"published": 1
}
"article2": {
"published": 0
},
},
},
}

What does this do? We have separated the attribute published and put it in another object with the same key but on a different list. Why? Let me explain using Firebase Rules. If we do the rules this way:

// Firebase Rules snippet"article_group": {
"article": {
"$article": {
".read": "data.parent().parent().child('article_list')
.child($article).child('published').val() == 1"
},
},
"article_list": {
".read": true,
}
}

We have separated the rules for reading the list and each article. We just have to add a little bit more from our original code to still get what we want. (You can see the working code here)

var articles = []firebase.database().ref('/article_group/article_list')
.orderByChild('published').equalTo(1)
.on('child_added', function(data) {
firebase.database().ref('/article_group/article/' +
data.key).on('value', function(articleData) {
articles.push({
id: data.key,
data: articleData.val()
})
})
})
})

What it means is that anyone can read the list of articles in the article_listand even if we can get the article_id of the unpublished Articles in our list, we will not be able to read them because it returns false from the rules under the article set. The key here is this:

"$article": { 
".read" : "data.parent().parent().child('article_list')
.child($article).child('published').val() == 1"
}
}

What it means is that it tries to check if the article_id has a published value of 1 in the article_list. Remember that the article_list is a sibling of articleunder article_group. Getting the data sets the query position to the current article, the first parent() sets the query position to article. The next parent() sets the query position to article_group. Calling child(‘article_list’) sets the query position to the sibling of article which is article_list. Going down further by calling child($article), which means it uses the key or id of the current article to set the query position on that object. Lastly, it checks the child attribute published to see if it is of value 1. If it is, then the read is successful, else it returns an error. (As you can see in this jsfiddle when we removed the equalTo).

Advanced Part: Sorting

Of course, blogs are sorted by date (let’s say date_created). We can change the data structure now to be:

{
"article_group" : {
"article" : {
"article1" : {
"body" : "This is the body of the article",
"date_edited" : 1333065600000,
"slug_name" : "this-is-an-article-abcdef",
"title" : "This is an Article",
"uid" : "user1"
},
"article2" : {
"body" : "This is a new body",
"date_edited" : 1333065600000,
"slug_name" : "article-on-firebase-hijklm",
"title" : "Article on Firebase",
"uid" : "user2"
},
"article3" : {
"body" : "This is a new body",
"date_edited" : 1333065600000,
"slug_name" : "article-on-firebase-hijklm",
"title" : "Article on Firebase",
"uid" : "user2"
}
},
"article_list" : {
"article1" : {
"published" : 1459296000000
},
"article2" : {
"draft" : 1333065600000
},
"article3" : {
"published" : 1461974400000
}
}
}
}

See that we removed the date_created part in each of the articles and changed the published value inside the items in article_list from 1 and 0 to date (in UTC milliseconds starting from midnight January 1 1970).

From this we can set the rules to be:

// Firebase Rules snippet"article_group": {
"article": {
"$article": {
".read": "data.parent().parent().child('article_list')
.child($article).child('published').exists()"
},
},
"article_list": {
".read": true,
}
}

We just need to check if the published attribute exists. (As an addition, you can check if it is greater than 0).

Then, we can change the code to accommodate the changes:

var articles = []
var limit = 2firebase.database().ref('/article_group/article_list')
.orderByChild('published').limitToLast(limit)
.on('child_added', function(data) {
if (data.val().published) {
firebase.database().ref('/article_group/article/' +
data.key).on('value', function(articleData) {
articles.push({
id: data.key,
published: data.val().published,
data: articleData.val()
})
if (articles.length == limit) {
sortArticle()
}
})
} else {
console.log('this is not published')
}
})
})// then after getting the necessary data
function sortArticle() {
article.sort(function(a, b) {
return a.published < b.published
})
// show article list in sorted
}

What did I do here? I added limitToLast, which gets the list from article_list in order of decreasing value. Based on this Firebase doc, it would still include those article_list items that don’t have the published attribute. But that would be the last one in the items to retrieve if we used limitToLast. Besides, we added a checker if it has the published value so that it wouldn’t necessarily download the bulk of the data which is in article.

If you have noticed, we still needed to sort it. Apparently, the Firebase event “child_added” doesn’t necessarily give you the data in decreasing sorted order. It will still give you in increasing sorted order but the last item is the max value and the first item retrieved would be the item in the n-limit place. Thus, we still have to sort it in decreasing order if we want to show it that way.

You can see a working version of that above here.

Final Thoughts

We have shown a different way of structuring the data for effective retrieval of filtered lists using Firebase. Next would be using this technique, along with Firebase roles, in creating, saving, publishing, deleting articles. Stay tuned.

If you have questions, clarifications or other things you would like me to talk about, please feel free to ask in the comments below.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store