Understanding Realm: Version Retention and Synchronization

If you’ve read my first article on Realm, you know that for each Realm.getDefaultInstance() call, there should also be a close()call in order to decrease Realm’s cache reference counter, and when the reference counter reaches 0, then the Realm is actually closed.

Typically I make sure I call Realm.getDefaultInstance()per thread only once, and on background threads I close them in a finally { block, on the UI thread I close them when my activity counter (incremented/decremented by retained fragment) reaches 0.

So you’ve learned that you must close the Realm, especially on background threads. But you might not know why it’s necessary. This article was written to explain that.

Realm’s MVCC architecture, and versions of data

Realm has this thing called an MVCC architecture (Multiversion Concurrency Control). It’s actually explained really well by Realm themselves in this article, and why it enables ACID and consistency and all kinds of good stuff. It’s actually a really good article, they should advertise it more.

— — — — — — — — — — — — — — — — — — — — — — — — — — —

The short version is that Realm is similar to a version control system like Git or SVN, where there are “revisions” or “versions” of a given data. When you use Realm.getDefaultInstance() for the first time, the Realm instance is “locked” to the latest version available in the Realm at that given moment in time.

If you recall, Realm is zero-copy (meaning whenever you obtain a managed RealmObject, it’s a proxy to the underlying database — but in reality, the database is accessed only when you access fields of the proxy, it doesn’t store the field values in memory). And that managed RealmObjects can only be written to in a transaction.

This is because when you’re not in a write transaction, then the Realm is locked to the latest version.

When you are in a write transaction, then you are directly creating the new version, seeing the new version you are creating at all times, rather than just being locked at a version. And when you do realm.commitTransaction() (well you generally don’t manually, use realm.executeTransaction() instead), this new version you created becomes “the latest version of the Realm”.

Version retention

The fact that you’re locked to a given version in order to enable consistency across the thread your Realm instance is on is called “version retention”.

However, it is important to know that a given version of the Realm exists as long as there is at least one Realm instance that still references that given version.

So in order to release an older version of the Realm, there are two things you can do:

1.) “refresh” the Realm, and synchronize to the latest version

2.) close the Realm

— — —

Well, this can lead to a problem. Background threads don’t auto-update (because they don’t have a Looper, which means Realm’s code cannot notify the Realm instance to synchronize itself to the latest version), and therefore they’re locked at a given version.

If you don’t close your Realm instance on a background thread, then the version retained by that Realm will exist for as long as the Realm instance exists.

@Override
protected void finalize() throws Throwable {
if (sharedGroupManager != null && sharedGroupManager.isOpen()) {
RealmLog.w("Remember to call close() on all Realm instances. " +
"Realm " + configuration.getPath() + " is being finalized without being closed, " +
"this can lead to running out of native memory."
);
}
super.finalize();
}

This is a problem, because old versions take up space. This space can be removed via compacting the Realm (when no thread has any open Realm instances), but if you don’t close the Realm, MANY versions will exist, and someone will always reference them.

That’s how you end up with Stack Overflow questions about “why is my Realm file 1 GB”, and why you should always close Realm in a finally {block on background threads (or in Swift, on the async dispatch queue, use autoreleasepool).

This is also why you probably shouldn’t do a transaction per saved object.

Synchronization

As I mentioned, the two ways to remove an old version of the Realm is by either closing the Realm (and freeing up resources), or via synchronization.

Manual synchronization ( refresh()) was removed in 0.89.0 although I think that was a mistake (and it was since re-added in 3.2.0!), but auto-refresh still works for looper threads (UI thread).

A Realm instance is updated by the following things:

  1. Local commits (realm.beginTransaction()), meaning synchronous transaction on that given thread
  2. Call to refresh()

On threads that have a looper and can/do auto-refresh (the UI thread), a message is sent to the looper indicating that there was a LOCAL_COMMIT. This forces the Realm to update on the next looper event, when syncIfNeeded() is called for the Realm instances, and then for every RealmObject and RealmResults that belongs to a Realm instance that belongs to that given thread.

In the case of transactions on background threads, in Realm 1.x.x and below, a REALM_CHANGEDmessage was sent to the looper directly with realm.commitTransaction().

In Realm 2.x.x, there is a Realm daemon thread, which sends the message when the Realm was written to. This is handled by the external commit helper, that allows multi-process notifications.

In Realm 3.x.x, this is handled entirely by the “object store”, and its RealmCoordinator.

And when the Realm receives this message, the synchronization occurs, similarly to local commits. The difference is that local commits and refresh() forces all asynchronous queries to run immediately (and therefore synchronously), while the other type of commit “waits” until all async queries are evaluated on their respective background threads. Then, the RealmChangeListeners are called.

Conclusion

Hopefully this helped you better understand how Realm works under the hood; and why you should always close the Realm instance (most importantly on background threads) — and maybe even better understand why you should query inside the transaction. All those things I mentioned in my first article — this explains why those tips are important.

One clap, two clap, three clap, forty?

By clapping more or less, you can signal to us which stories really stand out.