How to Use Go With MongoDB

Before jumping into the code, Let’s assume, I’d like to meet you, a brand-new application company that name is App-Doc. Let’s talk about a little bit story of App-Doc. The company has been released hundreds of applications in the world. App-Doc holds some information about application settings. That information related to application name, domain, IP address, country, etc…

In this post, I’d like to talk about some Go codes and the meaning of Mongo DB with screenshots and repositories. The main content is running Web Server that is working some requests and responses also communicating database. If we look deeply, you’ll learn about creating Go web applications and responsibilities about folders, writing Mongo DB queries with a CLI, writing docker file and docker compose file also working with mock data.

When I was starting to first Go Web Applications, I ask myself how to prepare the design in a single project. I searched for some Go project standards online and the books. I think all result almost comes the same. Then I decided to design a similar project layout. Other questions, which packages should be used it I asked myself. In this post I don’t talk about the history of Go, what is Mongo DB why do we use them. My purpose is implementation, required packages and usages also writing some queries.

Okay well, let’s jump start.

Pre-Requirements

Generally, I prefer to VS Code editor cause of any type of extension you can find it out. Also, install the Docker app.

Golang: https://code.visualstudio.com/docs/languages/go

Docker: https://www.docker.com/products/docker-desktop

Project Layout

When I searched the project standards, I found this GitHub repository which is https://github.com/golang-standards/project-layout. That repository has 27.7K stars however Go team has not released layout standards. However, if you search a layout type you can find out these

• Flat layout
• Group by Functional layout
• Group by Module layout
• Hexagonal layout

In this part, I’ll explain folder structures and meanings.

Firstly, every source Go, inside of the src folder. The src folder includes are cmd and pkg folders. The cmd folders have 3 other folders. The cmd file means commands so main.go file is inside the appdoc-server folder. The sample data is related to the dummy data insertion and the utils folder is responsible for configuration and some logical custom errors. Another folder is an internal folder that includes net/HTTP requests for example request another domain. Now let’s talk about pkg folders and responsibilities. The pkg folder is responsible for database connections and health checking(optional), request and response models, database operations, and web server that is some Web Server Framework.

Day 1: Working with GIN and Logrus

Before I write the code create a new project structure as below then jump into the project root. Now let’s run these commands.

go mod init github.com/bburaksseyhan/appdoc-api
go mod tidy

Now install the required package below.

go get -u github.com/gin-gonic/gin
go get -u github.com/sirupsen/logrus

curl http://localhost:8080/health

Day 2: Working with Logrus and Viper

go get -u github.com/spf13/viper

On that day I’m going to implement read configuration via Viper and log the information read from the settings field by field. I create 2 configuration files one of them config.yml file if run the application locally the other one is .env file that is running on the docker. By the way, main(), read_configuration(), and read() functions are signed private cause I only use in that file then fill the configuration.go. The read_configuration functions look at information firstly environment variable if doesn’t exist read config.yml file otherwise environment variable read. Both files result fill the same Configuration struct.

Now I’m going to log and display all Configuration struct field by field using log.WithFields().

curl http://localhost:8080/health

Day 3: Working with MongoClient

Every time initialize the application check the database status via ping function by itself. At the same time mongo.Client and client.Ping function required context. Context.Todo() means empty context that is used in the main or main thread but has a wider application. Today we are going to separate handler and routing so create a handler that is related to request and response result also request the repository. In the below, I’ll add the required packages.

go get -u https://go.mongodb.org/mongo-driver/mongo
go get -u https://go.mongodb.org/mongo-driver/mongo/options

I created an AppDocHandler interface that holds some functions which is take pointer gin.Context parameters.

In the next section, I’m going to add new functions like Add, Delete, Update, etc… At the same time, I created an appDocHandler private struct that holds only pointer mongo.Client. In the next day, we’ll add to the repository, so NewAppDocHandler is a public function and returns AppDocHandler interface what does it mean? NewAppDocHandler implements to server.go file and take the pointer client parameter and return the value of pointer client via appDocHandler. In server.go file call the health check function.

router.GET("/health", handler.Healthcheck)

router.Get function take 2 parameters one of them relative string path another one handlers.

Healthcheck(*gin.Context)

Request cancellation comes from the client if the 30 seconds do not respond then the timeout will be triggered.

ctx, ctxErr := context.WithTimeout(c.Request.Context(), 30*time.Second)

c.JSON()

{"data":"learning go"}

c.IndentedJSON()

{
    "data": "learning mongo"
}

As you see below, every function returns some JSON data currently used see JSON but in the next day, I’ll change the IndentedJSON.

Let’s call the ConnectMongoDb functions in the server.go file.

On this day we create a repository, handlers at the same time change and add new features. Now I’m going to explain new features function by function. Before we jump-start, I added the required packages as below.

go get -u go.mongodb.org/mongo-driver/bson
go get -u go.mongodb.org/mongo-driver/bson/primitive
go get -u go.mongodb.org/mongo-driver/mongo
go get -u go.mongodb.org/mongo-driver/mongo/options

I have already announced to we are working on MongoDB. I want to only do some of the implementations on that part don’t wait for all of them. Now create a new interface that name is AppDocRepository. That repository holds some of the functions of CRUD operations. One of the inputs here takes a context parameter for each function. The appDocRepository struct deals with receiver functions.

receiver function

The NewAppDocRepository function takes config and client parameters which are these come from the first initialize.

Now I’m going to implement each interface method and explain them. The repository file doesn’t include logs just return the error or success information then handlers handle it with them.

Add function

That function takes appDoc single entity and context after that connect to the database and get the parameterless collections then call the InsertOne function which includes the client and check the error handling.

List function

That function is related to all of the collection of inside data. The list is required to take parameters that parameters meaning of how many data counts take it from the database. We have been waiting list of data. Therefore, we use the cursor’s Next() function. The cursor visits line by line and appends the appDoc slice then returns them.

GetById function

The GetById function takes object_id parameters than the function response single row collection. The findOne functions required filter that is related for where=@parameters. Decode will unmarshal the document represented by this SingleResult into appDoc

Delete function

Delete function is although they have the same features of GetById functions, the Delete functions call the DeleteOne() func and return the DeletedCount that is int64 data type information.

From above we developed a database repository. We learned about receiver functions, connection creation, options, filters, read data from cursors, decode, multiple returns of functions. The next step is handlers. Handler meaning of HTTP request and responses. The next step is the handler. Handler meaning of HTTP requests and responses, reading parameters from the body, converting to model to entity or entity to model, custom HTTP status types, error handling. Handlers have the same implementation as a repository.

Looks good now I’m going to initialize with repository and handler on the server.go file.

repository := repository.NewAppDocRepository(&config, client)
handler := handler.NewAppDocHandler(client, repository, config)

Now changing the routes as a below. The API endpoints are grouped. It might be the API has authentication, role by access, or a newer version.

Let’s run the app and check the healtcheck status.

curl http://localhost:8080/api/v1/health

api/v1/health

Application Pattern Schema

application schema

Day 5: Working with dummy data

• Line 17: read data.json file
• Line 19.21: error handling read operations
• Line 23: close the os.Open() operation
• Line 25: create a new AppDoc empty slice
• Line 27: convert data json to byte array
• Line 30.32: unmarshall byte array, fill appDocs and handled if error throw
• Line 37: connect to MongoDB client
• Line 40: close the connection with an empty context
• Line 42.44: get database and collections and check the database status
• Line 52.54: visit row by row collections and insert row

Before we run the insert code, please remove the Id field in both model and entity files otherwise the program inserts empty object_id. Why do I decide to remove Id field? Every insertion result has been already creating a new objectId. Struct tags such as json:"appdoc" specify what a field’s name should be when the struct’s contents are serialized into JSON. Without them, the JSON would use the struct’s capitalized field names – a style not as common in JSON.

Entity:

Model:

Day6: Prepare docker & docker-compose file

I create a docker-compose file and that file has 2 services. MongoDB has its own health-check command and the API depends on MongoDB's running status. Also, our API gets environment variables from .env file and open the 8080 ports.

Day7: Writing custom response models

In our app has some endpoints you have already seen it. Although I’d like to design the same pattern for all responses. Therefore I created our custom basic response go files. In our ResponseError struct includes some fields that are related to Message, Status, Error, and Data. Here is the most important field is Data because I declare map[string]interface{} type what does it mean? The main concept is any developer returns different values and I don’t want to write a field for everything. If we want a collect any type of thing, each one identifying. I just described the schema of JSON objects.

ResponseError

Response

At the end of Day7, I merged all branches to the main branch then change some codes, fields and solve the bug fixes. So what did I change? Let’s look at the handlers. At the end of the day, I changed response models and clear some extra codes. You can look the more codes in my repository.

appdoc_handler

docker-compose up 

Mongo CLI and Queries

docker container ls
docker exec -it container mongo

Now I connect to CLI. Let’s write some mongo queries.

show dbs // display all existing databases
use AppDb // switch the database
show collections // display all existing collections
db.applications.find({}).pretty()
db.applications.find({"_id":ObjectId("61a4ac47561122a1f7537dfe")})

Test a REST API with curl

List Function:

curl http://localhost:8080/api/v1/appdoc/list/100

Get Function:

curl http://localhost:8080/api/v1/appdoc/get/61a4ac47561122a1f7537dfe

Delete function:

curl http://localhost:8080/api/v1/appdoc/get/61a4ac47561122a1f7537dfe

At the end of the day I’ll share the repository URL and here is the AppDoc company some information below. If you want to find more data you should visit my repository.

Repository: https://github.com/bburaksseyhan/appdoc-api.git

Summary,

As I said at the top of the content, we only assume, we just imaginary create it. App-Doc is not a real only imaginary but I just mentioned different ideas writing Web API. Maybe I thought it might be more enjoyable. I was starting the learn Go languages, I found out about some books and documentations. These are very helpful, and it is important for practice. Here are, my recommendation books. Thank you for giving a time and reading.

• Go Programming Language (Alan Donovan)
• Learning Go (Jon Badner)
• Go In Practices (Matt Butcher)