Building APIs in Go beyond Hello World

My current minimal approach when building production ready APIs in Go (golang).

Update: Part Three

Update: Part Two

You should Unit Test your API endpoints, it’s easy!

TL;DR
Straight to the Source code then https://github.com/oliversavio/youtube-vid-code/tree/main/go-api-starter

Let’s get started right away, some of the things I use are as follows:

• A minimal web framework, for this post I’ve choosen gofiber
• A way to read configurations at runtime. The approach of using ENV variable has grown on me, we’ll use a lib called godotenv to achieve this.
• A way to (self)document the API, swagger is a popular choice and our web framework integrates well with it. I use swaggo/swag and arsmn/fiber-swagger.
• A way to build, package and deploy the API, Docker will be the tool of choice here. Check out the video below for more details how and why I use Docker.
• A way to manage the project on my development machine as well as on remote servers. This is where make will help keep things tidy.

Development workflow with Docker

Tools needed on development system

• Docker CE
• make
• Go 1.16
• swaggo/swag

Project Dependencies

go get github.com/gofiber/fiber/v2
go get github.com/swaggo/swag/cmd/swag
go get github.com/arsmn/fiber-swagger/v2
go get github.com/joho/godotenv

Project Structure

This is a highly opnionated topic, in general the Go convention is to keep things as simple and clear as possible. Here’s what works for me.

├── Dockerfile
├── Makefile
├── config.go
├── errors
│   └── custom.go
├── go.mod
├── go.sum
├── handlers
│   └── handler.go
├── main.go
├── middlewares
│   └── fiber_builtin.go
└── routes
    ├── public.go
    └── swagger.go

The Makefile

The Makefile is used to build, run, stop the Docker container and generate the swagger documentation. It also contains some default parameters which may be overriden as per the runtime enviroment.

ENV=development
VERSION=latest
LOCAL_PORT=9000
docker.build:
 docker build -t oliversavio/api:$(VERSION) .
docker.run:
 docker run --name api_test -d \
  -e FOO_ENV=$(ENV) \
  -p $(LOCAL_PORT):80 \
  --log-driver local \
  --log-opt max-size=10m \
  --log-opt max-file=5 \
  oliversavio/api:$(VERSION)
docker.clean:
 docker container rm api_test
docker.stop:
 docker stop api_test
swag:
 swag init
.PHONY build: swag docker.build
.PHONY run: docker.clean docker.run

An important point to note is the use of the “log-driver local” switch when running the Docker container, this uses a filesystem based log store with log rotation.

Commands

make build    
make run      

The Dockerfile

I use multi-stage Docker builds to test, build and create the standalone Docker container to execute the API. One advantage of this approach is that I only have to configure Docker on remote servers.

This approach plays nicely with the CI/CD mechanism provided by Gitlab and GitHub, however I will not go into it in this post, you may view my previous video on the topic linked below.

from golang:1.16 AS builder
LABEL MAINTAINER="Oliver M"
WORKDIR /src/
COPY . .
RUN go fmt $(go list ./... | grep -v /vendor/) &&\
    go vet $(go list ./... | grep -v /vendor/) &&\
    go test -race $(go list ./... | grep -v /vendor/) &&\
    GOOS=linux go build -a -o bin/app *.go
FROM debian:buster-slim
WORKDIR /api/
COPY --from=builder ["/src/bin/app", "/src/.env*", "/src/docs", "/api"]
EXPOSE 80
CMD ["./app"]

The Code

I strive to keep this as minimal as possible.

The main Function

This is where I initialise and configure the web framework aka gofiber in this case, configure the middlewares and routes. Then I start the server and listen for termination/interrupt signals, this is essential to implement a graceful shutdown of the application.

Loading ENV and Configuring GoFiber

I have merely included the loadEnv() function mentioned in the godotenv documentation here. IMHO the convention on handling configuration across environments works wonderfully.

Note that fiber has been configured with a custom error handler, we’ll go into the details later in this post. GoFiber and Echo have a nice mechanism of handling errors centrally.

The approach we use here will work for Echo as well.

Server Start and Graceful Shutdown

In order to implement graceful shutdown for GoFiber, I create a channel which will get notified whever there is an interrupt, typically invoked via CTRL+C. The server listener is started on a seperate go routine and the main thread waits for shutdown interrupts as shown in the section above.

In the shutdownGracefully() function, I’ve created a timeout and invoked the server shutdown. This ensures either a successful shutdown or a forceful termination of the server. In the defer func() is where other resource like database connections may be released.

Custom Error Handling

By defining a custom error type, I’m able to propagate specific error responses from either my handler or service code and also have a standard response to unexpected / unhandled errors.

An example from handler.go

handlers/handler.go

routes/public.go

Swagger Documentation

To get started with Swagger documentation, the main() function and the handlers need to have the swagger augmentations via comments. Next we need to ensure the swaggo/swag tool is installed on the development system. If you look at the Makefile in the section above, you’ll notice I have defined a “swag” target, which has been chained to the “build” target. This ensures we generate the most up-to-date documentation whenever there is a change in our code.

This is what the main function looks like now. You may explore the documentation at https://github.com/swaggo/swag to see all the possible annotations available.

main.go

handlers/handler.go

Two essential steps for getting the swagger documentaion to work are
- Adding import _ “oliversavio/api/docs” to the main.go
- Setting up the Swagger route.

routes/swagger.go

Now, invoking a make build run should generate the documentation as shown below.

Generated Swagger Documentation

Further Reading

There are a lot of advanced topics like monitoring, alerting, distributed tracing etc that are often needed for production application which I have not covered here. Some of the tools I use and encourage you to explore are:

• Prometheus for monitoring and alerting.
• Jaeger for distributed tracing.
• uber-go/zap for structured logging.

Part Two

You can follow along with the next part in this series here. This goes into a bit of code refactoring in order to make our API endpoints easier to unit-test.