Golang Microservices — New generation programming language

By Krishnaraj Ramakrishnan

Introduction

As part of a technology-driven organization, it is certain that the expectations grow exponentially higher every time for long-term sustenance. So, there is always a demand for solutions that offer high sustainability, less maintenance, high performance, low cost, and extended scalability. Though being part of the JAVA shop for a long duration, it has its own limitations and disadvantages that include high infrastructure cost, extensive coding, large development, and testing turnaround time especially with high-end workstations. Considering all these challenges, research was performed to identify other lucrative options as replacements that can work well with the SaaS-based Microservices architecture.

Thus, Golang, which is a statically typed, compiled programming language, was explored and tried out in some of the projects. The results appeared very promising and futuristic. The advantages experienced ranged from the ease of adaptability for programming, and scalability to the reduction of server costs and container size (for example 1 GB to approximately 50Mb) which also helped in speeding up the build and deployment time significantly.

It is always difficult to break the status quo to explore new possibilities that can provide us with more efficient solutions. To take that decision and venture into new arenas over the established was never easy, but eventually, when it yields the intended results, it was all worth the effort.

Considering the project requirements, some of the existing frameworks such as Gin and Krakend gateway were explored and finally decided to go with a customized code structure.

Golang based microservices framework and folder structure

The following miniature version of the structure hosted on GitHub, will be referred going forward.

Fig 1.1

Fig 1.1 has 3 major components.

• A Gateway — The request go through the Gateway which does the authentication and authorization for all the services and enforces rate limiting, enables tracing, so on..
• An Account service — The service does the actual authentication, authorization and the account management (managing tenants, users, role permissions, so on..)
• Product service — Product catalog service that is created for demo purposes.

Now let’s take an example with a high-level workflow. All the requests are funneled through Krakend (Now called Lura) based API gateway. As mentioned earlier, all the APIs are first authenticated using an account service. This framework also provides a common utility (micro-common). A separate code repository for the utility services is maintained. It is mandatory for any business common functionality that cuts across microservices should be added to this micro-common codebase.

The approach mentioned here may sound mundane to someone skilled in the art, but this certainly will be a good starting point for someone who is looking to venture into the Golang project. It takes care of several aspects of the project.

Now, let’s peek into the folder structure of individual microservices.

The key takeaway from the above structure

app- contains the application start-up logic and bootstrap functions.

module- contains all the application’s business logic. Developers need to touch only the module files. Each module contains

• router.go — handles all the API routing logic
• hander.go — handles all the HTTP requests and responds along with payload validation
• service — handles all the business logic
• repo — handles all the database operations and makes use of gorm for DB abstraction
• model — handles the ORM(Object Relational Mapping) objects and the request/response objects

doc — stores the auto-generated swagger docs.

locale — stores the language texts of the application.

config — maintains application level configurations. Most of the configurations are retrieved from the .env file when the application boots up.

migration — holds all the database migration scripts.

Makefile — contains the commands required to run the project.

cmd — acts as the entry point for the application.

deploy — contains the files related to application deployment into a docker container and kubernetes.

Readme.md contains the steps to setup and start each service.

The project extensively uses the gin framework and gorm for the ORM library. It uses Makefile for running commands. Same structure is followed by other services as well.

Features supported by the project

Fig 1.2 — Framework features

Run the service

Follow the steps to run the application

Step 1: Package Installation

Go to the service folder and download the package. eg catalog-service

cd catalog-service
go mod vendor

Step 2: Configuration

Change the configuration in .env for database

Step 3: Database migration

Run the below command to setup the database schema and data. All database tables setup and related data contains in the migration folder. It uses goose library for database migration.

make migrate-up

To create a new migration file and make some database changes

make migrate-create NAME=create-product-change

Step 4: Swagger doc

Generate API document to the doc folder using swag command.

make doc

Step 5: Start the service

make run

Note: may have to run “make migrate-up” before executing this command.

Step 6: Testing

Generate the mock files that are required for testing the application.

make mock

to run the test

make test

The same steps can be followed by other services (e.g.:- account service) as well.

Once completed all the steps, production-ready code can be deployed to the container-based application.

Krakend — High performance gateway

Krakend is a high-performance open-source API Gateway. Its core functionality is to create an API that acts as an aggregator of many microservices into a single endpoint, doing the heavy lifting automatically. It can aggregate, transform, filter, decode, throttle, authenticate, and so on.

Krakend uses a configuration file called krakend.json for configuration. Users can read more detail about krakend configuration file here.

Configuration

Some important configurations

• endpoint — defines the API mapping.
• extra-config — enables different components for the gateway

Build Krakend locally

To build the Krakend locally, it is required to create the binary locally rather than running the prebuild instance to utilize the plugin feature and custom container changes.

Steps

1. Download source code from https://github.com/devopsfaith/krakend-ce/
2. Go to source folder and run make build
3. Copy the Krakend binary to go bin folder {GOPATH}/bin

Copy the Krakend binary to {GOPATH}/bin path once created.

Note: was using the tag v1.3.0 at the time, now it is 2.x

Building the Krakend plugins

There are several types of plugins. Reference given here. Router plugin is enabled as mentioned in krakend.json.

Steps to build plugin

cd plugins/router-plugin
make build

Generate Krakend Config from dynamic configuration

Go to the parent folder and build the krakend-out.json using the below command

cd ../..
make build

User should make sure that the current directory is “gateway” (./gateway) and not the plugin directory(cd ../../).

The build process will generate a krakend-out.json file with final configuration from the templates and settings.

Start the API gateway

The configurations can be changed in Makefile as desired

PERMISSION_URL=http://localhost:8082/api/v1/authorize
ACCOUNT_SERVICE=http://localhost:8082
PRODUCT_SERVICE=http://localhost:8083

Run the gateway

make run

If there are any configuration changes to krakend.json, the user would need to rebuild the configuration using the “make build” in gateway folder. Refer to the Makefile to understand the various commands that are used for various processes.

Using the application

Once the swagger documentation is generated for product and account service using “make doc” command, user can access the following URLs

Account Service — http://localhost:8080/account/swagger/index.html

Product Service — http://localhost:8080/product/swagger/index.html

How to use the APIs

Following steps are to create a tenant and user, then accessing any of the API using authorization token.

Invoke the /tenantRegister api in account service to create a tenant and user.

Curl request

On a successful execution, a tenant and a user will be created in database.

Generate token for admin

Invoke the /adminLogin API to generate an access token which can be further used in the subsequent API calls. This token is generated using the user email and password that are used while creating an account.

There is another API called /token for normal users.

Curl request

Response

Any subsequent API calls need an access token returned in this response. It is a JWT token and follows the Oauth2 bearer token standards.

Product creation API

For invoking any apis user needs to pass the access token which is generated using the /adminLogin API. This value is passed into the respective API in the Authorization header (Authorization: Bearer {token})

If the API is invoked successfully, it means that the user has set up the services successfully.

Quick recap.

• Setting up an account and product service.
• Setting up the API gateway.
• Setting up an account and login for the admin part of the application.
• Invoking a sample product API with custom authentication.

Some of the advantages identified after using the Golang based framework are

1. Improved productivity — Feature development time has significantly reduced as the code is much simpler and highly maintainable.
2. Building and testing the code is much faster with minimal system specifications. Previously, it used to take 10 to 20 min to build and deploy and now its significantly reduced to 1–2 minutes.
3. Higher performance and infra cost reduction due to less CPU and memory footprint. The production costs are drastically reduced by 50%, which also helps in accommodating more containers in a single machine (Node).

Though still in the early phase of the transition, the aim is to continue exploring and gradual migration of the rest of the services that are running in Java to Golang.

Feel free to join Us if you are also interested to be a party to it.

Reference

Krakend Framework — https://www.krakend.io/

Code reference — https://github.com/krishnarajvr/microservice-mono-gin-gorm

Gin Framework — https://github.com/gin-gonic/gin