A fullstack epic part I — a REST API in Go accessing Mongo DB

Developing the the backend side of a dummy app

TL;DR;

This is a detailed tutorial of how to create a very simple RESTful API in Golang. If you want to grab the full code at once, you can come here.

Introduction

When we work in IT Engineering today, we can notice there is a lot of stuff coming out every day.

And, of course, we cannot be an expert of everything! We need a set of technos that “seduce” us more than others. Some of us feel more comfortable working with front-end technologies, others prefer to stay in the back/Ops.

However, it is extremely important that we know a little bit of the trending in different domains. This helps us having the skill to actively participate in decision making, challenging the different propositions that are put on the table. The best professionals I have met had a word on every layer of a software to be delivered, even if they were “out of their box”.

And I strongly believe that, the best way to really getting in touch with anything, is giving it a little try. This is the goal of this series of stories. I wanted to see how I can do the steps of developing and delivering/scaling a simple webpage. I am going to do it in three parts:

• a REST API in Golang, accessing data from a Mongo DB instance
• a simple page — which will evolve afterwards — in Angular (done from Angular CLI)
• delivering and scaling everything via Docker/Kubernetes

Let us start with the back-end side: the REST API in Go

The extremely simple dummy musicstore API

We are going to develop a web site that will display a single page with an inventory of music albums. Here we are going to develop the API that will retrieve the data from the DB and forward it to the page.

Setup

We start installing Go and Mongo DB. (just follow the instructions on the links).

I have chosen Go because it is an extremely simple -however powerful- language. It is easy to learn, and has an already strong community (you can find some links in the end of the post).

For our small implementation, Mongo is perfect. A NoSQL easy-to-use DB with the flexibility of managing documents, and very light. Strongly recommend it if you don’t want headaches with DB part.

In any case, feel free to work in any techs you feel comfortable and confident with.

In order to make Mongo work, you need to start its server in a terminal (or a prompt if you are in Windows) and leave it open:

# Start Mongo server
$ mongod 

Another option is to run it as a daemon, using the --fork option (like this you can run the rest of the commands in the same terminal)

# Option 2: Start Mongo server as a daemon
$ mongod --fork --logpath <path-to-logs>

Insert dummy data (optional)

If you want to preseed the DB, create a JS file, e.g. insert.js, and paste the following contents into it:

use musicstore;
var bulk = db.albums.initializeUnorderedBulkOp();
bulk.insert(   { title: "OK Computer", artist: "Radiohead", year: 1997, id: 1 });
bulk.insert(   { title: "The Queen is dead", artist: "The Smiths", year: 1986, id: 2 });
bulk.insert(   { title: "Be Here Now", artist: "Oasis", year: 1997, id: 3 });
bulk.insert(   { title: "Appetite for Destruction", artist: "Guns N Roses", year: 1987, id: 4 });
bulk.insert(   { title: "Back To Black", artist: "Amy Winehouse", year: 2006, id: 5 });
bulk.insert(   { title: "Hotel California", artist: "Eagles", year: 1976, id: 6 });
bulk.execute();

Then, run this command in a terminal:

# inserting items
$ mongo < <your-file>.js

OK, our DB is ready. We can develop our API. Just pick the Editor of your choice. I have chosen Visual Studio Code.

The Golang REST API

In your $GOPATH/src folder, create a subdirectory called ‘musicstore’. It should have the following structure:

src
|-- musicstore
|   |-- logger
|   |   |-- logger.go
|   |-- album
|   |   |-- controller.go (where we are going to place the handlers)
|   |   |-- model.go (the object representation)
|   |   |-- repository.go (DB access)
|   |   |-- router.go (the routes definition)
|   |-- main.go
|   |-- Dockerfile (for part III)

Then, we are going to need external libraries for routing and connection to Mongo DB. Type these commands in a terminal:

# getting gorilla/mux libraries for routing
$ go get github.com/gorilla/handlers
$ go get github.com/gorilla/mux
# getting mgo library for handling Mongo
$ go get gopkg.in/mgo.v2

Now, we shall write our code. First, the entrypoint: main.go

// main.go
package main
import (
 "log"
 "net/http"
"github.com/gorilla/handlers"
"musicstore/album"
)
func main() {
 router := album.NewRouter() // create routes
// these two lines are important in order to allow access from the front-end side to the methods
allowedOrigins := handlers.AllowedOrigins([]string{"*"}) 
 allowedMethods := handlers.AllowedMethods([]string{"GET", "POST", "DELETE", "PUT"})
// launch server with CORS validations
 log.Fatal(http.ListenAndServe(":9000",
  handlers.CORS(allowedOrigins, allowedMethods)(router)))
}

If you take a look in the bold comments, I needed to do some configuration in order to allow CORS access. Without this, the webpage will not be able to access the resources from the REST API.

Now, we define the routes, in album/router.go

// album/router.go
package album
import (
 "net/http"
"musicstore/logger"
"github.com/gorilla/mux"
)
var controller = &Controller{Repository: Repository{}}
// Route defines a route
type Route struct {
 Name        string
 Method      string
 Pattern     string
 HandlerFunc http.HandlerFunc
}
//Routes defines the list of routes of our API
type Routes []Route
var routes = Routes{
 Route{
  "Index",
  "GET",
  "/",
  controller.Index,
 },
 Route{
  "AddAlbum",
  "POST",
  "/",
  controller.AddAlbum,
 },
 Route{
  "UpdateAlbum",
  "PUT",
  "/",
  controller.UpdateAlbum,
 },
 Route{
  "DeleteAlbum",
  "DELETE",
  "/{id}",
  controller.DeleteAlbum,
 }
}
//NewRouter configures a new router to the API
func NewRouter() *mux.Router {
 router := mux.NewRouter().StrictSlash(true)
 for _, route := range routes { 
  var handler http.Handler
  handler = route.HandlerFunc
  handler = logger.Logger(handler, route.Name)
router.
   Methods(route.Method).
   Path(route.Pattern).
   Name(route.Name).
   Handler(handler)
}
 return router
}

Here we put all the routes in a list. Like this, if we need to add/delete a route, we’ll just need to modify the Routes list.

Next: the album model, representing an object from our DB:

// album/model.go
package album
import "gopkg.in/mgo.v2/bson"
//Album represents a music album
type Album struct {
 ID     bson.ObjectId `bson:"_id"`
 Title  string        `json:"title"`
 Artist string        `json:"artist"`
 Year   int32         `json:"year"`
}
//Albums is an array of Album
type Albums []Album

This is the structure we are going to send/receive from external parties.

Next: the repository (DB handling)

// album/repository.go
package album
import (
 "fmt"
 "log"
"gopkg.in/mgo.v2"
 "gopkg.in/mgo.v2/bson"
)
//Repository ...
type Repository struct{}
// SERVER the DB server
const SERVER = "localhost:27017"
// DBNAME the name of the DB instance
const DBNAME = "musicstore"
// DOCNAME the name of the document
const DOCNAME = "albums"
// GetAlbums returns the list of Albums
func (r Repository) GetAlbums() Albums {
 session, err := mgo.Dial(SERVER)
 if err != nil {
  fmt.Println("Failed to establish connection to Mongo server:", err)
 }
 defer session.Close()
 c := session.DB(DBNAME).C(DOCNAME)
 results := Albums{}
 if err := c.Find(nil).All(&results); err != nil {
  fmt.Println("Failed to write results:", err)
 }
return results
}
// AddAlbum inserts an Album in the DB
func (r Repository) AddAlbum(album Album) bool {
 session, err := mgo.Dial(SERVER)
 defer session.Close()
album.ID = bson.NewObjectId()
 session.DB(DBNAME).C(DOCNAME).Insert(album)
if err != nil {
  log.Fatal(err)
  return false
 }
 return true
}
// UpdateAlbum updates an Album in the DB (not used for now)
func (r Repository) UpdateAlbum(album Album) bool {
 session, err := mgo.Dial(SERVER)
 defer session.Close()
 session.DB(DBNAME).C(DOCNAME).UpdateId(album.ID, album)
if err != nil {
  log.Fatal(err)
  return false
 }
 return true
}
// DeleteAlbum deletes an Album (not used for now)
func (r Repository) DeleteAlbum(id string) string {
 session, err := mgo.Dial(SERVER)
 defer session.Close()
// Verify id is ObjectId, otherwise bail
 if !bson.IsObjectIdHex(id) {
  return "NOT FOUND"
 }
// Grab id
 oid := bson.ObjectIdHex(id)
// Remove user
 if err = session.DB(DBNAME).C(DOCNAME).RemoveId(oid); err != nil {
  log.Fatal(err)
  return "INTERNAL ERR"
 }
// Write status
 return "OK"
}

Note: our webpage only displays/creates albums for now. The Update and Delete stay there for next evolutions of our front part.

The logic of DB is quite simple: we open a session to Mongo DB, we perform the operation(s) and then we close the connection.

// opens the connection
session, err := mgo.Dial(SERVER)
 if err != nil {
  fmt.Println("Failed to establish connection to Mongo server:", err)
 }
// closes the connection before leaving the function
 defer session.Close()
// perform operations

Now we only need to handle the calls to our routes. Time to develop controller.go:

// album/controller.go
package album
import (
 "encoding/json"
 "io"
 "io/ioutil"
 "log"
 "net/http"
 "strings"
"github.com/gorilla/mux"
)
//Controller ...
type Controller struct {
 Repository Repository
}
// Index GET /
func (c *Controller) Index(w http.ResponseWriter, r *http.Request) {
 albums := c.Repository.GetAlbums() // list of all albums
 log.Println(albums)
 data, _ := json.Marshal(albums)
 w.Header().Set("Content-Type", "application/json; charset=UTF-8")
 w.Header().Set("Access-Control-Allow-Origin", "*")
 w.WriteHeader(http.StatusOK)
 w.Write(data)
 return
}
// AddAlbum POST /
func (c *Controller) AddAlbum(w http.ResponseWriter, r *http.Request) {
var album Album
 body, err := ioutil.ReadAll(io.LimitReader(r.Body, 1048576)) // read the body of the request
 if err != nil {
  log.Fatalln("Error AddAlbum", err)
  w.WriteHeader(http.StatusInternalServerError)
  return
 }
 if err := r.Body.Close(); err != nil {
  log.Fatalln("Error AddAlbum", err)
 }
 if err := json.Unmarshal(body, &album); err != nil { // unmarshall body contents as a type Candidate
  w.WriteHeader(422) // unprocessable entity
  if err := json.NewEncoder(w).Encode(err); err != nil {
   log.Fatalln("Error AddAlbum unmarshalling data", err)
   w.WriteHeader(http.StatusInternalServerError)
   return
  }
 }
 success := c.Repository.AddAlbum(album) // adds the album to the DB
 if !success {
  w.WriteHeader(http.StatusInternalServerError)
  return
 }
w.Header().Set("Content-Type", "application/json; charset=UTF-8")
 w.WriteHeader(http.StatusCreated)
 return
}
// UpdateAlbum PUT /
func (c *Controller) UpdateAlbum(w http.ResponseWriter, r *http.Request) {
 var album Album
 body, err := ioutil.ReadAll(io.LimitReader(r.Body, 1048576)) // read the body of the request
 if err != nil {
  log.Fatalln("Error UpdateAlbum", err)
  w.WriteHeader(http.StatusInternalServerError)
  return
 }
 if err := r.Body.Close(); err != nil {
  log.Fatalln("Error AddaUpdateAlbumlbum", err)
 }
 if err := json.Unmarshal(body, &album); err != nil { // unmarshall body contents as a type Candidate
  w.Header().Set("Content-Type", "application/json; charset=UTF-8")
  w.WriteHeader(422) // unprocessable entity
  if err := json.NewEncoder(w).Encode(err); err != nil {
   log.Fatalln("Error UpdateAlbum unmarshalling data", err)
   w.WriteHeader(http.StatusInternalServerError)
   return
  }
 }
success := c.Repository.UpdateAlbum(album) // updates the album in the DB
 if !success {
  w.WriteHeader(http.StatusInternalServerError)
  return
 }
w.Header().Set("Content-Type", "application/json; charset=UTF-8")
 w.Header().Set("Access-Control-Allow-Origin", "*")
 w.WriteHeader(http.StatusOK)
 return
}
// DeleteAlbum DELETE /
func (c *Controller) DeleteAlbum(w http.ResponseWriter, r *http.Request) {
 vars := mux.Vars(r)
 id := vars["id"] // param id
if err := c.Repository.DeleteAlbum(id); err != "" { // delete a album by id
  if strings.Contains(err, "404") {
   w.WriteHeader(http.StatusNotFound)
  } else if strings.Contains(err, "500") {
   w.WriteHeader(http.StatusInternalServerError)
  }
  return
 }
w.Header().Set("Content-Type", "application/json; charset=UTF-8")
 w.Header().Set("Access-Control-Allow-Origin", "*")
 w.WriteHeader(http.StatusOK)
return
}

Basically, we handle the request variables (if any), call the Repository to do the proper modifications in the DB, and return the Response to the caller.

[UPDATE]: Finally, the logger

Here we also log the inner calls to our API. Like this, we are able to trace which calls have been done to our API, as well as their parameters.

In your musicstore/logger/logger.go:

package logger
import (
    "log"
    "net/http"
    "time"
)
func Logger(inner http.Handler, name string) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        start := time.Now()
inner.ServeHTTP(w, r)
log.Printf(
            "%s\t%s\t%s\t%s",
            r.Method,
            r.RequestURI,
            name,
            time.Since(start),
        )
    })
}

Test it

We are done! Now we can do a small test. Open the terminal (be sure your Mongo server is running), and run the following commands:

# browse to the solution's directory
$ cd $GOPATH/src/musicstore
# launch it!
$ go run main.go

Then, open http://localhost:9000/ in your web browser, and you’ll see an empty list or the dummy data if you inserted it.

What now?

Our backend is ready, now it is time to build our webpage. This will be done in Part II.

If you want to jump over to the deployment/scaling part, go to Part III.

Useful links

Gopher Academy

Go Newsletter

Dave Cheney Blog

Go By Example