My Clean Architecture Go Application
I fully moved to Go stack a few months ago and suffered a lot with the good approach for architecturing the code and structuring the project. Went through multiple open source projects, talked to multiple Go devs and finally came up with some template and approaches I’m using for all new projects. Maybe this will be interesting to all who are struggling with the same problems!
I will go through creation simple account management API and try to apply my vision of clean architecture in Go applications.
What is a Clean Architecture
It’s the way of structuring your application to make it simpler to develop/extend/support and more understandable. Clean architecture usually related to some other things imho.
Good project structure
When your project structure is easy to understand and follow. When you know where every part of your code is located.
This is the link to my demo project on GitHub
Project consists of the following directories and files:
api — this is where all protobuf schemas and related generations are located ( including DTOs). This is related to adapters layer or our application. We have Makefile in this directory to split it out from the main Makefile, and prevent developers from regenerating API too often.
cmd — is where our main application is located.
infrastructure — where we have env related docker, env setup. Also later this is a good place to put your IaaC configs, helm files and other configurations for your environments.
internal — contains application code.
tests — for integration tests. I think having a single directory for integration tests helps here, since in a big projects it’s really hard to split out unit tests from integration tests.
Low coupling and high cohesion
Another key to clean architecture is low coupling between your layers, types, and high cohesion.
Application is mostly split into different parts/ layers. If these layers are low coupled then it’s better to develop/extend/support and test. They are simpler to understand (you don’t need to know about relations, but only about the current file/part/layer). Also layered architecture helps to put same stuff within a single layer and provide high cohesion, when all related stuff are well grouped.
In microservice based architecture we usually do split our code by domain. The example is covered only account domain. So we will architect our code structure split by layers.
API
API first approach
I decided to use API first approach during designing my application. API is a single source of truth and an interface to your system.
Code is changing frequently by many people, you might change something non intentionally and it could break your API. API if it’s designed as a single file/document has less chances to be non intentionally changed.
Since we are in Go I decided to write API with gRPC and protobuf, and after that generate related codes from it, and also for some external clients I will generate swagger. I won’t setup swagger here, you can use any OpenAPI client to read specification.
This is an example of autogenerated swagger file, you can view it on github
API is capable to handle grpc and http requests and based on grpc-gateway. grpc could be used later to communicate between your microservices in a bigger system, and http API could be used for external clients.
API is related to adapters layer of our architecture and contains DTOs we are using to interact with our system. DTOs aren’t related to domain model we have in the system and might be different.
Versioning
API should be versioned from the beginning, this could help to avoid problems with project architecture later. We start with 1st version as v1.
Domain
The core of our architecture — is our domain — the business area we are writing code to handle.
Domain models don’t depend on any other circles! But since it’s a core element of our architecture outer layers could reference domain models.
I see that many developers are choosing the way when they put VO, DB entities, DTOs, all interfaces between systems to the domain layer and make this layer shared within the whole application. I think this approach is bad in the same way as shared state is bad. Since DB entities are related only to repository layer (which is located inside adapters), then we cannot share access to them from the inner layers.
Same for handler DTO’s and interfaces. If we are using repository from service layer then we should place repository interface to the service layer as well — as nearer as possible to the consumer (service itself).
Our domain domain is based on Anemic Domain Model — models without behaviour, and all the behaviour, business rules and validators are moved to upper service layer.
Data first approach
The same approach as for API we are using for our data layer — we will specify our data directly in the database and use it as a source of through. This will help us to get rid of problems with breaking changes on data layer when you accidentally changed some struct. To migrate data we will use db migrations and initial scripts.
This also will help us in testing, since we could make our fixtures in tests, and this won’t affect our domain objects.
Use Cases
For me this is similar to command pattern — it should be responsible for some operation you’d like to execute on the system.
Use cases cannot interact with other usecases. But they could call multiple services which they need to perform their application logic.
For example if you need to update User’s role you might need to make a call to UserService and then you call UserRolesService.
Services
Services are located in the inner circle for use cases. The main idea for this layer — is to provide a glue between your application level business logic (use cases) and your enterprise level business logic (repositories). Service is a good place to have common logic shared between multiple use cases — this will avoid massive use cases and copy/pasting.
Services should not interact with other services and should be simple.
Services are organised around domain and domain-specific. In our example we will have just a single service — account related.
In this example services look like just a single wrapper around repository, but this application is very basic. In big projects/applications this service might contain some additional business logic related to your domain model.
Adapters
It’s the outer circle in our architecture. This is an interface level — here we have setup our controller — web/grpc server, setup our handlers and use this layer to connect to the database through repositories.
Controllers on this layer are adapters for incoming requests. Call from infrastructure layer goes directly to the controller layer. In this example controllers are split between API directory and internal/adapters but they are on the same architecture layer, so you could easily use grpc requests/response DTOs here.
Here we do all the request/response validations and also pass call to the use case which will handle it.
Response conversion and validation is also done on this layer.
Controller layer need to know about all use cases it could use for further requests processing.
Communication between layers
We need to support proper layering for all the code in our application. Requests and responses should become to controllers (adapters) layer and db entities should become to repository (adapter layer as well). if service layer requires access to repository — then related interface should be placed on service layer.
The main rule here — all relations should go inward — means outer layer knows about inner one, but not vice versa. We don’t push structs from outer layers to inner layers. Yes they are mostly have same data, but they might be different in behaviours.Structs should work only within it’s layer and should not be transferred to inner layers! Outside layers need just pure data and our goal to pass only raw/pure data from inside to outside.
This is known as “the depenency rule” — inner circle should know nothing about outer circle — this possible means that we should convert db entities to domain in repository (since repository could know about domain object, but service should not know about repository details!). By the same token, data formats used in an outer circle should not be used by an inner circle, especially if those formats are generate by a framework in an outer circle. We don’t want anything in an outer circle to impact the inner circles.
As I mentioned above — domain layer is shared between all layers.
Every layer should contain everything it needs!
Forward path of requests processing is always from outside to the domain. We receive request — process use case . If we need some persistence then do these changes with service layer.
But when we need to access outer circles we will do it via interfaces (low coupling). We will create interfaces only when needed and will create them on consumer side.
Testing
Every layer is independent in clean architecture and we could test it independently with unit tests. Integration test are placed to tests folder to split them up from unit tests (since they require infrastructure to be started).
Our Makefile contains up script which is used to start the application locally and apply db migrations and fixtures. This is a prerequisite for integration testing - we start standalone service along with database and next start our integration tests. up script contains live reloading functionality so we could develop main application in parallel with tests.
Let’s start
we are going to use buf to generate our API
here is our .proto definition
syntax = "proto3";
package me.sadensmol.article_my_clean_architecture_go_application.contract.v1;
import "google/protobuf/timestamp.proto";
import "google/api/annotations.proto";
import "protoc-gen-openapiv2/options/annotations.proto";
option (grpc.gateway.protoc_gen_openapiv2.options.openapiv2_swagger) = {
info: {version: "1.0"};
external_docs: {
url: "http://localhost:8081";
}
schemes: HTTP;
};
message Account {
int64 id = 1;
string name = 2;
AccountStatus status = 3;
google.protobuf.Timestamp opened_date = 4;
google.protobuf.Timestamp closed_date = 5;
AccessLevel access_level = 6;
}
enum AccountStatus {
ACCOUNT_STATUS_UNKNOWN = 0;
ACCOUNT_STATUS_NEW = 1;
ACCOUNT_STATUS_OPEN = 2;
ACCOUNT_STATUS_CLOSED = 3;
}
enum AccessLevel {
ACCOUNT_ACCESS_LEVEL_UNKNOWN = 0;
ACCOUNT_ACCESS_LEVEL_FULL_ACCESS = 1;
ACCOUNT_ACCESS_LEVEL_READ_ONLY = 2;
ACCOUNT_ACCESS_LEVEL_NO_ACCESS = 3;
}
message CreateAccountRequest {
string name =1;
AccessLevel accessLevel =2;
}
message CreateAccountResponse {
int64 id = 1;
}
message GetAccountRequest {
int64 id =1;
}
message GetAccountResponse {
int64 id =1;
string name = 2;
AccountStatus status = 3;
AccessLevel accessLevel =4;
google.protobuf.Timestamp opened_date = 5;
google.protobuf.Timestamp closed_date = 6;
}
service AccountService {
rpc Create(CreateAccountRequest) returns (CreateAccountResponse) {
option (google.api.http) = {
post: "/api/v1/account"
body: "*"
};
}
rpc GetById(GetAccountRequest) returns (GetAccountResponse) {
option (google.api.http) = {
get: "/api/v1/account/{id}"
};
}
}server setup is done in main.go
here we create a db connection, and setup required usecases and service/repository.
package main
import (
"context"
"database/sql"
"flag"
"fmt"
"github.com/sadensmol/article_my_clean_architecture_go_application/internal/account/adapters/controllers"
handler "github.com/sadensmol/article_my_clean_architecture_go_application/internal/account/adapters/repositories"
"log"
"net"
"net/http"
"os"
"github.com/golang/glog"
"github.com/grpc-ecosystem/grpc-gateway/v2/runtime"
"github.com/joho/godotenv"
_ "github.com/lib/pq"
gw "github.com/sadensmol/article_my_clean_architecture_go_application/api/proto/v1"
"github.com/sadensmol/article_my_clean_architecture_go_application/internal/account/services"
"github.com/sadensmol/article_my_clean_architecture_go_application/internal/account/usecases"
"google.golang.org/grpc"
"google.golang.org/grpc/credentials/insecure"
)
var (
grpcServerEndpoint = flag.String("grpc-server-endpoint", "localhost:9090", "gRPC server endpoint")
)
func main() {
flag.Parse()
defer glog.Flush()
err := godotenv.Load("infrastructure/local/.env")
if err != nil {
log.Fatal("Error loading .env file")
}
var connectString = fmt.Sprintf("host=%s port=%s user=%s password=%s dbname=%s sslmode=disable", os.Getenv("DB_HOST"),
os.Getenv("DB_PORT"), os.Getenv("DB_USER"), os.Getenv("DB_PASSWORD"), os.Getenv("DB_NAME"))
db, err := sql.Open("postgres", connectString)
if err != nil {
log.Fatalf("Failed to connect to database %v", err)
}
defer db.Close()
ctx := context.Background()
ctx, cancel := context.WithCancel(ctx)
defer cancel()
lis, err := net.Listen("tcp", ":9090")
if err != nil {
log.Fatalln("Failed to listen:", err)
}
s := grpc.NewServer()
accountRepository := handler.NewAccountRepository(db)
accountService := services.NewAccountService(accountRepository)
accountUsecases := usecases.NewAccountUsecases(accountService)
accountHandler := controllers.NewAccountHandler(accountUsecases)
gw.RegisterAccountServiceServer(s, accountHandler)
// Serve gRPC server
log.Println("Serving gRPC on 0.0.0.0:9090")
go func() {
log.Fatalln(s.Serve(lis))
}()
// Register gRPC server endpoint
// Note: Make sure the gRPC server is running properly and accessible
mux := runtime.NewServeMux()
opts := []grpc.DialOption{grpc.WithTransportCredentials(insecure.NewCredentials())}
err = gw.RegisterAccountServiceHandlerFromEndpoint(ctx, mux, *grpcServerEndpoint, opts)
if err != nil {
log.Fatalln("Failed to listen:", err)
}
gwServer := &http.Server{
Addr: ":8090",
Handler: mux,
}
log.Println("Serving gRPC-Gateway on http://0.0.0.0:8090")
log.Fatalln(gwServer.ListenAndServe())
}Db entities and related code is generated via go-jet and committed to this directory
please notice, we always should commit out generated code along with the project!
We have 2 usecases in our application
package usecases
import (
"github.com/sadensmol/article_my_clean_architecture_go_application/internal/account/domain"
)
func (a *AccountUsecases) Create(account domain.Account) (*domain.Account, error) {
return a.accountService.Create(account)
}package usecases
import (
"github.com/sadensmol/article_my_clean_architecture_go_application/internal/account/domain"
)
func (a *AccountUsecases) GetByID(ID int64) (*domain.Account, error) {
return a.accountService.GetByID(ID)
}they are just wrappers around account service which is used to interact with repositories
package services
import "github.com/sadensmol/article_my_clean_architecture_go_application/internal/account/domain"
type AccountService struct {
accountRepository AccountRepository
}
type AccountRepository interface {
Create(account domain.Account) (*domain.Account, error)
GetByID(ID int64) (*domain.Account, error)
}
func NewAccountService(accountRepository AccountRepository) *AccountService {
return &AccountService{accountRepository: accountRepository}
}
func (s *AccountService) Create(account domain.Account) (*domain.Account, error) {
return s.accountRepository.Create(account)
}
func (s *AccountService) GetByID(ID int64) (*domain.Account, error) {
return s.accountRepository.GetByID(ID)
}package repositories
import (
"database/sql"
"fmt"
"github.com/go-jet/jet/v2/postgres"
"github.com/sadensmol/article_my_clean_architecture_go_application/db/gen/account/public/model"
"github.com/sadensmol/article_my_clean_architecture_go_application/db/gen/account/public/table"
"github.com/sadensmol/article_my_clean_architecture_go_application/internal/account/domain"
)
type AccountRepository struct {
db *sql.DB
}
func NewAccountRepository(db *sql.DB) *AccountRepository {
return &AccountRepository{db: db}
}
func (a *AccountRepository) Create(account domain.Account) (*domain.Account, error) {
var savedAccount model.Account
err := table.Account.INSERT(table.Account.Name, table.Account.AccessLevel).
VALUES(account.Name, account.AccessLevel).
RETURNING(table.Account.AllColumns).Query(a.db, &savedAccount)
if err != nil {
return nil, err
}
return RepositoryModelAccount(savedAccount).toDomain(), nil
}
func (a *AccountRepository) GetByID(ID int64) (*domain.Account, error) {
var savedAccounts []model.Account
err := table.Account.SELECT(table.Account.AllColumns).
WHERE(table.Account.ID.EQ(postgres.Int(ID))).
Query(a.db, &savedAccounts)
if err != nil {
return nil, err
}
if len(savedAccounts) == 0 {
return nil, nil
}
if len(savedAccounts) > 1 {
return nil, fmt.Errorf("GetByID returns non unique result")
}
return RepositoryModelAccount(savedAccounts[0]).toDomain(), nil
}entry point is the controller on the adapters layer
package controllers
import (
"context"
"google.golang.org/protobuf/reflect/protoregistry"
"google.golang.org/protobuf/types/known/timestamppb"
"log"
contractv1 "github.com/sadensmol/article_my_clean_architecture_go_application/api/proto/v1"
"github.com/sadensmol/article_my_clean_architecture_go_application/internal/account/domain"
)
type AccountHandler struct {
accountUsecases AccountUsecases
}
type AccountUsecases interface {
Create(account domain.Account) (*domain.Account, error)
GetByID(ID int64) (*domain.Account, error)
}
func NewAccountHandler(accountUsecases AccountUsecases) *AccountHandler {
return &AccountHandler{accountUsecases: accountUsecases}
}
func (h *AccountHandler) Create(ctx context.Context, request *contractv1.CreateAccountRequest) (*contractv1.CreateAccountResponse, error) {
log.Println("create was called!!!")
account, err := h.accountUsecases.Create(domain.Account{
Name: request.Name,
Status: domain.AccountStatusNew,
AccessLevel: AccountHandlerAccessLevel(request.AccessLevel).ToDomain(),
})
if err != nil {
log.Fatalf("Error occurred %v", err)
}
return &contractv1.CreateAccountResponse{Id: account.ID}, nil
}
func (h *AccountHandler) GetById(ctx context.Context, request *contractv1.GetAccountRequest) (*contractv1.GetAccountResponse, error) {
account, err := h.accountUsecases.GetByID(request.GetId())
if err != nil {
log.Fatalf("Error occurred %v", err)
}
if account == nil {
return nil, protoregistry.NotFound
}
return &contractv1.GetAccountResponse{
Id: account.ID,
Name: account.Name,
Status: AccountStatusFromDomain(account.Status),
AccessLevel: AccountAccessLevelFromDomain(account.AccessLevel),
OpenedDate: timestamppb.New(account.OpenedAt),
ClosedDate: func() *timestamppb.Timestamp {
if account.ClosedAt != nil {
return timestamppb.New(*account.ClosedAt)
} else {
return nil
}
}(),
}, nil
}This approach helps me to make applications quick and maintain/support later without any problems.
Please let me know if you’re interested in next part of this story! And also put list of questions you want me to cover there!
Thank you for reading.
Further plans for part 2 (if somebody is interested)
- add context based transactions support
- move to better http error handlers
- add linters and .editorconfig
- integration with .devcontainers
- add proper logging and monitoring
- refactor api into pkg folder and make it accessible via external lib
- add additional microservices and show integration between them
- add ability to debug main application during the test
Resources
More information about protobuf
Buf cli to generate grpc services and swagger openapi docs
Really nice library inspired by Java’s JOOQ (yes I love it and miss in Go world)
Grpc gateway allows you to have http adapter on your GRPC service
And this is a good article on Medium I took for initial inspiration:
