A Non-Intrusive Transaction Management Lib in Go — How it Works?

4 min readJul 18, 2020

In my article “A Non-Intrusive Transaction Management Lib in Go — How to Use It”, I described how to use the transaction lib. In this article, I will focus on the Lib itself. Some of you may have read “Go Microservice with Clean Architecture: Transaction Support”, which described the old version of the transaction management system. There will be some overlap between that article and this one. Since most people probably haven’t read that one or forgot what it is about, so it is better to start from scratch (Assume that you haven’t read the previous one).

A good transaction lib should be transparent to the application who uses it. In Go’s “SQL” lib, there are two types of database connection, “sql.DB” and “sql.Tx”. When you don’t need transaction support, use “sql.DB”; otherwise use “sql.Tx”. To share the persistence layer code for both scenarios, we need a database wrapper to support both. I got the idea from “db transaction in golang”

The interface for the database layer

The database layer is the lowest in the lib which handles database access. The business application doesn’t need to deal directly with that layer, only the transaction management lib needs to do that.

The database access wrapper

The following is the shared database access interface for both transactional and non-transactional operations. The interface is defined in “gdbc.go”.

It has two parts. One is the database interface that supports the normal database operations such as query tables, update table records. The other (Transactioner) defines the functions to support the transaction, such as “commit” and “rollback”. The “SqlGdbc” interface is the combination of the two. The interface will be used to connect to the database.

The implementation of the database access interface

The following is the code implementation of the normal database operations. It is in file “sqlConnWrapper.go”. It defined two structs, “SqlDBTx” is a wrapper to “sql.DB”, which will be used by non-transactional functions; “SqlConnTx” is a wrapper to “sql.Tx”, which will be used by transactional functions.

Implementation of transaction

The following is the code implementation of “transactioner” interface. It is in file “txConn.go”. I got the idea from “database/sql Tx — detecting Commit or Rollback”

Because “SqlDBTx” doesn’t support transaction, all the functions for it have a null implementation.

The interface for the persistence layer

On top of the database layer is the persistence layer, which is used by the business application to manipulate records saved in database tables. You need a function to enable transaction in this layer. The following is the transaction interface for the persistence layer and it is in “txDataService.go”.

The following is the implementation of it. It just calls the underline database function of “TxEnd()”, which has been implemented in the database layer. The code is not part of the transaction lib (It is the only code in this article that is not in the transaction lib), and you need to implement it in your business application.

The code to get the database connection

Besides the interface we described above, you need to get the database connection before doing anything else. There are two functions to do it.

The function returns “SqlGdbc” interface

Function “Build()” (In “factory.go”) will return “SqlGdbc” interface. Based on the parameter you passed in, the underline struct it returned will be either “SqlConnTx” for transaction or “SqlDBTx” for non-transaction. Function “Build()” is the easy one to use if you don’t need to get hold of the underline database connection.

The function returns database connection

Function “BuildSqlDB()”(In “factory.go”) will return “*sql.DB” and ignore the transaction flag parameter that you passed in. After you get the database connection, you need to generate either “SqlConnTx” or “SqlDBTx” yourself based on the transaction flag. If you to cache “*sql.DB” in your application, then you have to use this one.

Limitations

First, it only supports transaction for the SQL database. If you have a NoSql database, it won’t work (Most NoSql databases don’t support transaction anyway).

Second, if your transaction boundary across databases (for example, among different Microservices), then it won’t work. The common idiom for that is to use “Saga Pattern”. You write a compensation action for each action in a transaction and execute the compensation action one by one during the rollback phase. It should not be difficult to add Saga solution in business applications. Should Saga be part of the transaction lib? That is an interesting question. Right now, I would think Saga should be in a separate Lib if you want to create one.

Third, it doesn’t support the nested transaction, so you need to manually make sure nested transaction is not happening in the code. If the codebase is not too complex, this is easy to do. If you have a complex codebase with a lot of transactional and non-transactional code intervened, then it is time to implement a nested transaction or find a solution supporting it. I didn’t take time to look into how much effort it takes to add the nested transaction, but it may not be trivial. If you are interested in it, “database/sql: nested transaction or savepoint support” is a good place to start with. So far, for most use cases, the current solution probably strikes a good balance between the effort and the reward.

How to expand

The “SqlGdbc” interface doesn’t list all the functions in the package “sql”, only the ones I needed in my application. You can easily expand the interface to include other functions.

For example, if you need to expand distributed tracing into the database, you may need to pass in the context into the database functions. The “sql” lib already supports database functions with context. You just need to find those and add them to the “SqlGdbc” interface and implement them in “sqlConnWrapper.go” and then in your persistence layer, you need to call the function with the context as a parameter.