Go-Room: Ninja
An advanced user’s guide to Go-Room
This is the last article in my series on Go-Room. I strongly recommend reading primer for Go-Room in practice and architecture deep dive of Go-Room as prerequisites. Go-Room is data store agnostic by design. If you are storing data on edge devices and would like to version control it’s schema, Go-Room can be an excellent choice. This article is intended to help developers channel their inner Ninja and plug Go-Room to any edge data store of their liking.
So what’s different?
In previous articles GORM was the point of reference for data access and storage. GORM is a widely popular ORM targeted at relational databases. It supports a wide variety of SQL dialects like postgres, mysql and sqlite. In this article we will take a look at a data store that resides at the other end of the spectrum. We will be using Go-Room to version control a BoltDB data store.
BoltDB is a fast and lean key-value store for go-lang applications. It is currently maintained by etcd-io. I can see Kubernetes fans twitching already :). The idea is to drive home the data store agnostic nature of Go-Room by presenting an integration example that is polar opposite of GORM.
The Fictional Edge App
For the purpose of this case study, we consider a fictional edge app that stores serialised data in BoltDB. The app serialises the data as JSON and stores raw bytes in BoltDB.
BoltDB data organisation is based on buckets. A bucket is a collection of related key value pairs and is identified by a unique key.
In our example application buckets are equivalent to tables and the contained key value pairs are analogous to rows.
Source Code
The source code for this case study is live on the go-room-examples repository. I would suggest you to explore the relevant parts code base as you progress through the article from here on.
Model Definition
The key to version management of schema is Model Definition generation support by the ORM. In this example the data is serialised as JSON and then stored as raw bytes. Hence the model definition here is the anatomy of the model as seen by a JSON Marshaller.
The model name is used as the key for the corresponding storage bucket in BoltDB.
In order to demonstrate version management across multiple versions of the app I have created individual packages to capture the schema at each such version of the app. Refer to the entity.go files in db package to take a peek.
Refer to the implementation of ORM interface here to understand the Model Definition rendered by our example application.
Migrations
Migrations for the fictional app can be found in migrations.go files in db package. At this point I would like to emphasise that the schema and migrations from all versions have been put in separate packages for demonstration purposes. In practice your app code would carry just the latest schema and all supported migrations.
Let’s take a look at the migration for version 1 to version 2. The change in schema in here is that User entity has a JSON tag that overrides the serialised storage key for Name field from ‘Name’ to ‘username’. This requires us to load all the stored JSON data for User entity and store it back again after updating the key for Name field.
Keen eyed readers would have noticed a Bolt Tx object being retrieved here. BoltDB requires any access/update to a bucket to be governed by a transaction. Hence migration scripts need a Tx object that can access buckets. This is where the data store agnostic ORM interface kicks in.
The ORM interface requires developer to implement a GetUnderlyingORM method which returns the object that migration scripts can operate upon. This ensures that your migration scripts can be as fancy as supported by the your data store. Hence, Go-Room does not limit the kind of DB operations you do during migration.
The Run Script
At this stage of your reading into the article I hope you now have a clear idea of how the model definition and migrations are laid out. The go-room-examples executable is intended to be run as a command line application.
Every run of the application will attempt to initialise a BoltDB database in a file named ‘goroomex.db’. It takes version number of schema as a mandatory argument from command line. As you run the script successively for higher version numbers Go-Room performs migrations as required.
The script also dumps the stored data to standard output by default after every initialisation so that you can visualise the effects of the migrations as you run through higher versions.
Fin
It has been a delight to work on Go-Room as my first open source project. I hope I was able to shed light on my work through this blog series.
Looking forward to your questions and feedback. If you like my work do Star Go-Room on github.
Thanks for reading. :)
