gokv: Go database interface

A TDD-approved persistence-layer abstraction

Kiwi provided by vecteezy.com

SQL is hard. Dealing with persistence in general is hard and error-prone. Sometimes the functionality one needs is very little over CRUD: in such cases, it may be interesting to have a library deal with the database-specific instructions.

ORMs are here for simplifying the process of wiring our code to a persistence layer, but most of the time they come with a fundamental flaw: they propose themselves as a generic implementation that will work for this and that database.

Instead of providing a generic implementation, gokv tries a different approach: it aims at defining a generic interface.

Defining a Store interface

The gokv/store.Store interface defines a basic set of high level methods, starting from the basic Set, Get, Update, Delete.

Not every store implementation will, nor should, implement every defined method. A Redis store will easily implement SetWithTimeout, but it might not be a goal for it to implement FindWord (still in the works, not pictured above). The consumer code will define a subset of this interface only featuring its needed methods.

In the PostgreSQL implementation, a Store represents a database table. It is instantiated with a *sql.DB and a table name. In Redis, a Store might represent a namespaced group of entries, possibly spread across different types.

The Add behaviour is defined as: a method that persists a new key-value pair, erroring if the key is already present.

The interface tells nothing about how the implementation will comply to this.

Moreover, the interface tells nothing about the tradeoffs. The same behaviour might be implemented very efficiently in PostgreSQL, and less efficiently in Redis… or vice-versa. The implementation’s documentation is expected to provide insight. The bright side is: benchmarking the different stores with your specific use case might be a matter of changing a couple lines of code.

Go K-V. Well I’m not actually sure it renders in English. Anyhow. Kiwis provided once more by vecteezy.com

JSON as an interchange format

The gokv interface is based on JSON marshalability. In order to avoid reflection and type assertion, the methods for persisting new data receive json.Marshaler, and the methods for fetching data accept json.Unmarshaler.

The methods for fetching multiple results leverage the Collection interface, defined as:

type Collection interface {
    New() json.Unmarshaler
}

In this slice-based implementation of a Book collection, New appends a new Book to the slice and returns a pointer to it as a json.Unmarshaler:

The Store implementation, while looping through the results from the DB, will unmarshal them into the Unmarshaler returned by New. Here is how the Collection is used in the gokv/postgres implementation of GetAll:

Consumer code is easy to test

Test-driven developers will find another advantage to gokv: the responsibility of dealing with the actual (possibly external) persistence is totally on the Store implementation. This means that on the consumer side, we can test every single line of code without any painful mocking. An example of 100% test coverage:

Looks cool! How can I contribute?

The project is currently little more than an experiment.

Implementations of the gokv interface exist for in-memory storage and PostgreSQL. A Redis implementation is in the works and I’m planning on a Redis-stream store. None of them is production-ready. The gokv/store interface itself lacks many important method definitions.

If you want to get involved, get in touch! Or just file a pull request. My main focus is now on defining sane interfaces for searching, but you might have another idea?