Go-Room
A version management utility for edge data stores
If you have written an application which runs on edge devices it is very likely that you used a data store. It could be to store user preferences, transaction logs or cached intelligence data for offline capabilities among many other such use cases. As developers focus on offline capabilities to address network issues and low bandwidth limits, data stores tend to be a crucial part of the application.
Android Room
SQLite is the weapon of choice in Android. A fast, reliable and yet hassle free database option in terms of contribution to application file size. It is through my work with SQLite on Android that I read about Room. It is an excellent utility that provides a very sleek interface for SQLite database creation and management.
Why make Go-Room?
I always wanted to read the source code for Android Room and understand its internals. Recently as I was working on an edge device application in go-lang I had to use a SQLite database. As a Room user from Android ecosystem I looked for a parallel option but to my joy :) I found none. So I set out on this project to make one.
Version Management
The major challenge that you face with a data store on edge apps is version management and migration. Edge apps are often updated at the convenience of the user and network. This results in a fractured ecosystem where multiple versions of your app exist in edge devices at a given point in time. A data store only adds to the complexity when you factor in changes to schema across various versions of the app.
The following figure demonstrates how different sections(by percentage) of users/edge devices can have different versions of your app as you release new versions over time.
Migration Pains
The biggest risk that comes along is your data migration queries feel radioactive as you need to account for migration from every possible version of your app ever released. This leads to the risk of data corruption and loss. A simple strategy would be to perform destructive migrations on every update but data stored on edge device could be too valuable to lose. For instance they could be transaction logs of a payment gadget that are yet to be synced to the server.
How it works?
Room in Android defines the whole interaction with database. It is intended to be declarative where a user declares their intention and Room generates a standard/safe implementation at build time.
In this project my intention was to replicate the version management part while abstracting the data access layer. A typical usage of go-room would be as follows.
The following code snippets will illustrate the use of Go-Room on a database using the excellent GORM for DB access.
The snippet below shows an adapter I wrote to use Room in one of my projects. The focus is on the InitGormDB method.
We can now break down each argument passed to the InitGormDB function to understand the usage steps I mentioned before.
db
It is the ORM object created by the app for database access.
entityList
This is a list of entities(tables) that are expected to exist in the database. The following snippets show the schema of entities in oldest and latest app versions respectively.
migrations
These are all the migrations rolled out with this app till date. Hence ideally Room is equipped to migrate the DB to the target version from any source version across a fractured ecosystem. In the gist below you can see the migrations defined for consecutive updates.
Room creates a graph to select the path for update from a given base to a given target version. This path could involve applying multiple migrations if required.
versionNumber
The expected Target Version in which the database must be for the App to function. This is a constant that is a part of your source code. You are expected to change it whenever you make changes to the schema.
fallbackToDestructiveMigration
A flag that forces recreation of database in case corruption or lack of migration path to target is detected.
Boilerplate
There are two other components that need to be initialised for Room.
gormAdapter
Implements standard ORM interface for GORM to function with Room. This ORM interface allows Room to perform clean up and version management.
identityHashCalculator
Implements a Hash calculation interface that allows Room to generate a unique stable signature for a database consisting of a given set of entities. This signature is coupled with version and serves as a straightforward check of compatibility.
In the snippet below you can see the final method call for initialising a GORM DB using Room.
When does Init fail exactly?
A database initialisation with room fails in following scenarios:
- You make changes to schema and fail to update the versionNumber.
- You make changes to schema and update the versionNumber but do not provide a migration. An empty migration is required even if you do not expect any database DDL to be run.
- You mess around with the boilerplate components. *With great power comes great responsibility*
What next?
In this article I covered a basic usage pattern of Go-Room with GORM. As I mentioned earlier in the article, Go-Room is agnostic to the data access layer. In plain English you can use a different ORM of your choice or even a different kind of data store (lets say a file storage) with Go-Room.
I intend to illuminate these points subsequently as I discuss further in my next two articles:
Part-2 -> “Go-Room: The Internals”
Part-3 -> “Go-Room: Advanced Usage”
Hope this was a succinct primer to Room. Looking forward to questions and comments from you :). Thanks for reading!
