Simple (And Easy) Data Persistence in Go

Blockchains are great, but sometimes you need off-chain storage

Credit: yucelyilmaz — Getty Images

If you’re coming from frameworks like Rails or Django, you’re using to having an ORM layer that seems to work like magic. But eventually you’ll run into issues as your app scales, when more power and finer grained control are needed.

With Go, it’s easy to build a custom persistence layer, giving you the best of both worlds — a simple API, and the power to harness advanced database features as apps scale.

A common best practice in creating a solid persistence layer is to abstract out the database driver. This allows flexibility in swapping between Postgres, MySQL, MongoDB, etc., without getting locked into any one platform.

When designing anything, a product, a software module, you always want to think about the interface first. Who will be the consumers of this abstraction layer? What is the simplest API they want?

At a basic level, a database provides reads and writes to data. Lets create a high level type for that:

The Datastoreinterface is our end goal, featuring two embedded types, Mutations, and Queries. It’s a further abstraction over the Repository pattern popularized by Martin Fowler. It gives us a clean separation of concerns by breaking out reads and writes, allowing better integration with technologies like GraphQL.

Now lets go back to the beginning, and see if we can arrive back at this interface.

The first step is wrapping a handle to the database client. We are using https://github.com/go-pg/pg in this case.

Great! Now what can we do with this client? The first thing is probably creating a table. The advantage of a strongly typed language like Go is that we can use the compiler’s knowledge of types to automate the grunt work of creating database-specific types and fields. For example, here we create a Postgres table directly from a Go struct:

RegisterModel will automatically create a table, translating Go types into Postgres fields. The pg module does most of the heavy lifting.

What’s next? Let’s add a row to the database:

Just like creating a table, the database driver is smart enough to translate Go structs into the appropriate fields needed to add a row to Postgres. And we get batch adds for free, once again, thanks to pg.

So far, both functions are made to operate on a generic Go structs, using the interface{} type. Let’s pull them into an interface:

I added a Remove method to round out the API.

Let’s do something similar for reads:

In the real world, these primitives are probably not going to be enough. They’ll take you a long way, but eventually you’ll have app-specific needs for your mutations and queries.

For example, you might have a query that selects on a field other than the primary key:

Or a mutation that’s specific to your app’s logic:

Let’s combine these with our previous generic functions to come up with a clean interface. Go’s embedded types comes in handy again here:

Do these interface names look familiar? We have finally arrived at our original API:

This interface can now be passed along to any piece of code that needs access to the database, such as GraphQL resolvers. It can also easily be mocked out for testing.

Where to go from here?

If your Queries interface gets too big. You can always make specific interfaces for each model, and embed them all into the main Queries interface. This way, each model will be individually testable.


Does building clean abstractions in Go and Typescript excite you? Do you want your code to have a positive social impact? Come work with me at TruStory.

TruStory is a trustless platform for validating and verifying content.