Writing My First Microservice Using Go
My first interaction with Go
A few months ago, I decided to learn a new programming language. I’m already familiar with C#, JavaScript (Browser and Node.JS), and SQL. After a few hours of googling, I decided to learn Go. In this article, I want to describe my first microservice written using Go, what libraries I’ve used, what challenges I had, and much more.
Introduction
If you’re unfamiliar with Golang, here is a brief overview of what it is.
Go is an open source programming language supported by Google. It’s easy to learn and get started with. It has growing ecosystem of partners, communities, and tools.
The biggest companies using Go are Google, Netflix, Dropbox, Uber, Meta, Twitch, etc. With Go, you can create cloud & network services, web applications, and command-line interfaces.
Before I started, I tried to find the benefits of using Go and here they are:
- it’s really easy to use. It has only 25 keywords available to use;
- it’s fast, faster than C# or JavaScript;
- it’s statically typed. You can check types on compilation time;
- it’s a young language, younger than C#, Python, Java, JavaScript;
- it has great community support. You can find whatever you need.
At this moment, I can’t highlight any disadvantages because everything Golang criticized for is made intentionally. For example, error handling significantly differs from C#, where you need to specify a try/catch block for exception handling. In Go, you’re returning an error as a function result as the latest parameter, and if it’s not absent — the error happened. But it makes you use errors even if you don’t want to (if you’re using return parameters — you must assign all of them to variables or explicitly ignore them).
Until Go 1.18, it didn’t support generics, but now you can use them if you need them. Still, Go does not support default parameters and function overload. It was also made intentionally to make reading code and understanding the call stack simple. So for me, it’s hard to find some pitfalls in Go.
How have I learned Go?
When I met new technology and I’m interested in it, I try to find a book that I can use for learning. This time I’ve bought a book by Jon Bodner “Learning Go: An Idiomatic Approach to Real-World Go Programming”.
You can find it on Amazon using this link. It is a great book that describes core Golang features in detail, helps you to set up a local environment for Go development, and shows best practices and patterns using Golang.
Another source of knowledge I’ve used is the official Go website. You can find excellent documentation of core Go libraries, how to use Go, some conventions, and much more.
The very first Go Microservice idea
As a backend developer, I’ve decided to develop a microservice using the Go language to try it out. The idea was to create a microservice that you can use to share passwords using some URL. API should provide the ability to save the password and receive an HTTP link that the other user can use to get the password saved previously.
I’ve created the following functional requirements for it:
- it should be scalable both horizontally and vertically;
- it should be robust and resilient to any failure;
- data should be persistent;
- it should have logs to be able to trace HTTP requests;
- it should have metrics to understand load, latency, and other critical indicators;
- it should be testable using autotests;
- it should encrypt sensitive data (password);
- CI should be present;
A pretty great list, but microservices must ensure at least these requirements to be possible to support them on production.
Infrastructure
As you can understand, I’ve decided to use Go to build the application layer. But to cover previous functional requirements, I need a database (to persist data), logging system (to collect logs), metrics database (to collect metrics), load balancer (to try to scale application horizontally), and solution for service discovery (to observe application state & health).
I won’t describe all benefits stack I’ve used because it’s a long story. I will give just a little introduction to every tool.
For the database, I’ve decided to use the Postgres database because it’s my favorite database. It was interested how to work with it using Golang. Nowadays, Postgres supports relational and document models, so it’s elementary to use Postgres for many purposes.
To collect logs, I’ve used ELK stack, which stands for Elasticsearch, Logstash, and Kibana. As well, to collect logs from a file, I needed Filebeat. It’s a great solution to aggregate, collect, store and visualize logs from your applications.
I’ve used Prometheus to collect metrics and Grafana to visualize data collected from the application.
To balance incoming traffic, I’ve used HAProxy because it’s easy to use and configure.
I’ve used HashiCorp Consul for service discovery because it helps monitor your application health, know where the application is hosted, and much more.
To create the infrastructure I need, I’ve used Docker and Docker Compose to be able to store infrastructure as a code (IaaS) and quickly create everything I need for local development.
For CI purposes, I’ve used GitHub Actions to build and test the code I’ve committed.
Let’s go further!
Application side libraries and frameworks
To handle HTTP requests, I’ve decided to use the Gin framework. It has excellent documentation, is easy to use, and is fast. What more do I need for my web application? Nothing!
For logging, I’ve used Zap developed by Uber, which helps you to collect structured logs that can be indexed further to Elasticsearch.
I’ve also decided to try ORM to integrate with Postgres, and I’ve used Gorm this time. You can do a lot of stuff using this library, and like previous libraries — it has rich documentation.
Other libraries I’ve used are not so interesting to describe here, but you can find everything I’ve used and source code in the repository below.
Application code
I won’t describe the whole development process (you can find the code using the link below). I better describe the biggest challenges I had during implementation:
The very first challenge I had was writing tests. I found some mock libraries, but they seemed a little bit hard for me, and I’ve decided to implement component tests instead of unit tests. It was a great decision because I tested many services using a single test. Of course, this approach has several drawbacks, but in my case, it seemed perfect.
Let me show one of tests that generate link from the password:
package service
import (
"context"
"testing"
"github.com/google/uuid"
"github.com/misikdmitriy/password-sharing/config"
"github.com/misikdmitriy/password-sharing/database"
"github.com/misikdmitriy/password-sharing/helper"
"github.com/misikdmitriy/password-sharing/logger"
"github.com/misikdmitriy/password-sharing/tests"
)
func TestCreateLinkFromPasswordShouldDoIt(t *testing.T) {
c := &config.Config{}
c.Database.ConnectionString = "inmemdb"
c.Database.Provider = "sqlite"
c.Encrypt.Secret = "123456789123456789012345"
c.Encrypt.IV = []byte{1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16}
ctxt := context.Background()
c.App.LinkLength = 8
loggerFactory := logger.NewTestLoggerFactory()
encoder := helper.NewEncoder(c)
dbf := database.NewFactory(c, loggerFactory)
err := tests.MigrateDatabase(ctxt, dbf)
if err != nil {
t.Error(err)
}
rf := helper.NewRandomFactory()
s := NewPasswordService(dbf, c, rf, loggerFactory, encoder)
result, err := s.CreateLinkFromPassword(ctxt, uuid.New().String())
if err != nil {
t.Error(err)
}
if len(result) != c.App.LinkLength {
t.Errorf("expected password length to be %d but was %d", c.App.LinkLength, len(result))
}
}Here I’ve created all the dependencies I needed to create the link: database (for the test, I’ve used SQLite), encryptor (encrypt/decrypt was tested in another file), logger (I’ve created logger for tests only), and so on. I’ve decided to use stubs instead of mocks to implement the behavior I need, but for this case, the only stub I’ve used is a logger.
The second challenge I had was dependency injection. If you are familiar with C#, you know that DI is a built-in feature in the .NET platform. But in Go, another approach is used — you create all the dependencies in the main method using factory methods and pass them into another instance. It’s easy, but for me, this is what I had to get used to.
Here is one of the dependencies:
package helper
import (
"crypto/aes"
"crypto/cipher"
"encoding/base64"
"github.com/misikdmitriy/password-sharing/config"
)
type Encoder interface {
Encode(data string) (string, error)
Decode(data string) (string, error)
}
type encoder struct {
config *config.Config
}
func NewEncoder(config *config.Config) Encoder {
return &encoder{
config: config,
}
}
func (e *encoder) Encode(data string) (string, error) {
block, err := aes.NewCipher([]byte(e.config.Encrypt.Secret))
if err != nil {
return "", err
}
plainText := []byte(data)
cfb := cipher.NewCFBEncrypter(block, e.config.Encrypt.IV)
cipherText := make([]byte, len(plainText))
cfb.XORKeyStream(cipherText, plainText)
return base64.StdEncoding.EncodeToString(cipherText), nil
}
func (e *encoder) Decode(data string) (string, error) {
block, err := aes.NewCipher([]byte(e.config.Encrypt.Secret))
if err != nil {
return "", err
}
cipherText, err := base64.StdEncoding.DecodeString(data)
if err != nil {
return "", err
}
cfb := cipher.NewCFBDecrypter(block, e.config.Encrypt.IV)
plainText := make([]byte, len(cipherText))
cfb.XORKeyStream(plainText, cipherText)
return string(plainText), nil
}Here I’ve just introduced a public interface Encoder implemented in the encoder struct and fabric method NewEncoder that returns an instance that implements the current interface.
In Go, interfaces are implemented implicitly by implementing interface methods. It wasn’t easy to understand after the C# if you know what I meant, but I liked it.
You can find an example of the main package below:
package main
import (
"github.com/misikdmitriy/password-sharing/config"
"github.com/misikdmitriy/password-sharing/controller"
"github.com/misikdmitriy/password-sharing/database"
"github.com/misikdmitriy/password-sharing/health"
"github.com/misikdmitriy/password-sharing/helper"
"github.com/misikdmitriy/password-sharing/logger"
"github.com/misikdmitriy/password-sharing/server"
"github.com/misikdmitriy/password-sharing/service"
)
func main() {
appConfiguration, err := config.LoadConfig()
if err != nil {
panic(err)
}
appLogger := logger.NewLoggerFactory(appConfiguration)
encoder := helper.NewEncoder(appConfiguration)
databaseFactory := database.NewFactory(appConfiguration, appLogger)
randomFactory := helper.NewRandomFactory()
service := service.NewPasswordService(databaseFactory, appConfiguration, randomFactory, appLogger, encoder)
pgHealthCheck := health.NewPgHealthCheck(databaseFactory, appLogger)
server := server.NewServer(
appLogger,
appConfiguration,
controller.NewCreateLinkController(service, appConfiguration),
controller.NewGetLinkController(service),
controller.NewHealthController(pgHealthCheck),
)
if err = server.Run(); err != nil {
panic(err)
}
}Finally, it was not so hard but something I needed time to understand after switching to a new language.
Conclusion
My first interaction with Golang was a pleasure. The language was designed with a minimum number of features but enough to build modern applications. It also has excellent community support, so you can always find support for your issues. The first steps were a great challenge but exciting, so I will continue to learn Go and try to implement other stuff with it.
