Building a Simple Golang Serverless Recommendation System on ZEIT Now with FaunaDB

Photo by Manuel Geissinger from Pexels

In 10Clouds, we always try to check new opportunities. We’ve recently successfully expanded our mobile offer with Flutter development and full service with Firebase as a backend, but I decided to try a different serverless solution.

In this article, I’ll describe a process of creation of a recommendation system with the use of The Go Programming Language and FaunaDB on ZEIT Now Serverless Platform. This language and two tools were new to me, so I’ll also put my thoughts about using them.

Project Concept and MVP

The initial goal of this project was to create a small SaaS that will gather user likes/dislikes and gave items recommendation based on the choices of others. Then I started to strip that idea to make proper MVP out of it:

• First of all, I decided to only store users and items identifiers. This will make this project data agnostic, and after all, additional data isn’t required for a basic recommendation algorithm.
• The logic of algorithm was not a part of this exercise, so from the beginning, I assumed that I’d use one of many open source solutions.
• To make things easier, I also resigned from batch likes/dislikes updates.
• I decided that different clients will be distinguished with request Authorization header. I didn’t add any validation logic, but you can easily do this to make sure that only data from clients with correct tokens are stored to the database.

After this, I had a concept of SaaS MVP with five endpoints: add/remove like, add/remove dislike and get a recommendation for given user and was ready to pick a technology.

Programming Language

I decided to get familiar (because mastering is a very long process) with a new language. Since I’m in static typing team (Swift, Dart, duh!) I picked Go. It looked like a perfect fit for this small project — easy to learn and to write. Additionally, it reminded me of my university time, and my first applications were written in C++. Soon after, my assumptions were confirmed. It is indeed an “easy to learn but hard to master” language. Go syntax is limited and therefore, can be acquainted very quickly. Lack of classes, simple package management, and build in code formatter are other advantages. But with all of this Go is also not as failproof like other modern programming languages.

It is the language created for efficiency, performance, and multiprocessing. This is why passing pointers to values is a common practice. Undeniably quicker in run-time (using the pointer to value is way faster than making a copy of the original value) is also side-effects vulnerable. With language selected, the next step was to pick where to deploy the code written in Go.

Serverless Platform

In the mobile team, we already have experience in Firebase Functions, so I decided to try something different.

Digression: I’m not sure Golang is supported on Firebase Functions because on page Cloud Functions for Firebase there is a fragment „Functions can be written in JavaScript or TypeScript”. From the other side, Google Cloud Functions, which Firebase is using supports JavaScript, TypeScript, Python, and Go. End of Digression.

I also considered Amazon Lambda, but usually, I don’t want to take a sledgehammer to crack a nut. So I picked Now from ZEIT. Serverless computing means that the developer can focus on writing logic code and scaling and managing of resources are handled by a third party (cloud provider). It is usually a cheaper solution than a dedicated machine because we pay only for truly used resources. But it generates another class of problems. The one I faced is a database that can work cooperatively with serverless code.

Database

I had a very hard time picking a serverless database which is not Firestore from Firebase. Reading through some articles, I learned about FaunaDB and decided to give it a try. FaunaDB is a NoSQL database that has been created for the serverless application. Fauna company offers FaunaDB Cloud solution that has affordable pricing with free quota big enough to make some experiments. Additional advantages are Go driver and indexes that we can use for data query. If you’re looking for different solutions, take a look at MongoDB, Google Cloud Firestore, DynamoDB, or Amazon Aurora.

Implementation

This section will be more technical. We will download the required tools, setup environment, and then create code for routing. After that, we will configure the database, and write code for storing data there. In the end, we get data from the database and return recommendations. If you don’t want to follow the creation process with me, just skip to Summary section, where I will write my thoughts about used tools and achieved solution.

Tools

To start work on Recommendation as a Service experiment will need two additional tools that you probably don’t have on your device. First one is Now app with CLI support to manage your Now project. You can download it from official site. The second one is a binary distribution of Go available here. Please remember to follow installation instruction from the download page.

Accounts on ZEIT Now and FaunaDB site will be also required.

Environment

Now when we have Now (pun intended) installed, we can create our work environment. Head to your Go path (you can display it from the console using echo $GOPATH), then src and create directory github.com and [your_github_username] in it. Then inside [your_github_username] directory call

now init go recommendations-go from command line. This will create recommendations-go with example Go application. You will find structure listed on the screenshot below.

If you go to that directory and call now dev you will be able to run local emulation of this project.

NOTE: In the example code and project repository [your_github_username] will be replaced with my username kamilpowalowski.

Routing

It is time to write some Go code. We will create a function that will handle our routing. For example, when user will call a PUT on https://domain/api/users/kamil/like/jedi it will be redirected to correct method that will add a dictionary with key user_id and value kamil and key item_id and value jedi to the database. Routing will also validate provided values and HTTP Authorization header.

First of all, delete date.go file since we don’t need it and create index.go in api directory. Next change line module example-date in go.mod file to module github.com/[your_github_username]/recommendations-go/api.

For routing will use mux package from Gorilla, the golang web toolkit. Gorilla is a web framework for writing efficient server-side code, but we will use only a routing package from it.

To use Mux we have to create new new router router := mux.NewRouter() and then add handle functions. For example given above:

go
router.HandleFunc("/api/users/{user_id}/like/{item_id}", users.Like).
    HeadersRegexp("Authorization", "Basic [a-zA-Z0–9]{1,128}").
    Methods(http.MethodPut)

Each HandleFunc contains path (with variables defined in curly braces) and function that will be called if given route will be called. In our case it will be a function Like in package users. We can also validate our HTTP headers (authorization header with basic token which contains only basic letters and numbers with length 1 to 128 characters) here or limit calls to specific methods (only PUT).

Mux will also offer an option to validate input variables during evaluation. Changing /api/users/{user_id}/like/{item_id} to /api/users/{user_id:[a-zA-Z0–9]{1,64}}/like/{item_id:[a-zA-Z0–9]{1,64}} will add value check to user_id and item_id keys (letters and numbers from 1 to 64 characters).

With that knowledge we can add endpoints to handle:

• adding and deleting likes: PUT and DELETE on /api/users/{user_id}/like/{item_id}
• adding and deleting dislikes: PUT and DELETE on /api/users/{user_id}/dislike/{item_id}
• getting recommendations for given user: GET on /api/users/{user_id}/recommendations

Now Go support will call a Handler(w http.ResponseWriter, r *http.Request) function at the begging. So put your router creation code in that function and at the end call router.ServeHTTP(w, r) to handle given request.

Completed index.go code with some optimization looks like this:

kamilpowalowski/recommendations-go

Routing revisited

There is one thing that we have to change to allow Now to handle our newly added routes. In now.json file find

"routes": [{ "src": "/", "dest": :"www/index.html" }]

and add an additional entry to routes array to make it looks like the following:

"routes": [
    { "src": "/", "dest": "www/index.html" },
    { "src": "/api/(.*)", "dest": "api/index.go" }
]

With that change we redirected all calls that starts with /api too newly created index.go file. NOTE: you can change name key in the same file to (for example) recommendations-as-a-service.

Database

After signing in to FaunaDB Cloud dashboard, create a new database using the New Database button. I named my DB recommendations-as-a-service. Next, create a new class and name it entries. Additional fields can stay at recommended settings. After that operation, you should already see one index named all_entries. We will need two additional new indexes.

1. Pick entries as a source class, name index entries_with_token, and put token in terms.

2. Pick entries as a source class, name index entry_with_token_user_item, and put token, user_id and item_id in terms.

After that operations, you should have FaunaDB Cloud Console that looks similar to one presented on the screen below.

There is one thing to do on this dashboard. We have to generate an access key for our server. Go to Console Home (NOTE: don’t use “Keys” button visible on the screen above) and use “Manage Keys”. Create new key by selecting the correct database and Server role. You can provide a name that you like. Write generated key down. We will use it in the next step.

Environment variables

Now by ZEIT support two ways of handling secrets. For local development build, we can store them in .env file. For production, we have to add them using a command line interface. We will do both with DB key that we get from the previous step.

First of all, create .env file in the project directory. Then open it in an editor and put the line FAUNADB_SECRET_KEY=[your_fauna_db_secret] into it.

Then we have to add the same (or different) database key to the production environment. You can do this by calling now secret add faunadb-secret-key-recommendations [your_fauna_db_secret] from the command line.

We have to register this secret in our application. To do this, once more open now.json file and add additional lines to it:

"env": {
    "FAUNADB_SECRET_KEY": "@faunadb-secret-key-recommendations"
}

If your are here, already change how our project should be compile by changing

{ "src": "api/**/*.go", "use": "@now/go" }

to

{ "src": "api/index.go", "use": "@now/go" }

Completed now.json file will look similiar to this one.

Utils

Before we will write our logic we can also prepare some functions that can be reused in other files (like the creation of DB connector, and sending HTTP responses). Create a new directory called utils in api. Inside utils directory place utils.go file and put the code below in it:

kamilpowalowski/recommendations-go

As you can probably already see, I put there functions for sending JSON responses (SendJSON, SendSuccess) and errors (SendInternalServerError, SendNotFound), to extract client token from authorization header (ExtractToken) and create FaundDB connector (DBClient).

Create, read, update and delete

Ok, now we are ready to write our logic. First of all, we will focus on putting and deleting data from our database. In api create users directory and add three new files there: like.go, dislike.go and entry.go. first two files will contain actions called from index.go file. The third one will give us some abstraction on database operations. We will start with a third one.

Entry

To make writing objects easier I’ve created a struct Entry with the body below:

type Entry struct {
    Token string `fauna:"token"`
    UserID string `fauna:"user_id"`
    ItemID string `fauna:"item_id"`
    Value int `fauna:"value"`
}

FaunaDB library supports tags started with fauna where we can specify how given field will look like in database (it’s not common to use capitalized keys required for public fields in Go).

I don’t want to make you sleepy so I put a full code for entry.go , but to give you an idea how it looks like, here we have code that uses DB index we created before to get existing item:

client.Query(
    f.Get(
        f.MatchTerm(
            f.Index(“entry_with_token_user_item”),
            f.Arr{entry.Token, entry.UserID, entry.ItemID},
        ),
    ),
)

we provided a name of index and values that should be used in given get query.

Creating objects looks similar:

client.Query(
    f.Create(
        f.Class(“entries”),
        f.Obj{“data”: entry},
    ),
)

First, we provide the name of a class and data that we want to add to it and then we call query on database client.

Full code of entry.go is available under entry.go.

Like and Dislike

Equipped with that knowledge we can build our handle methods. Since we extracted most of the database code to the separated files like method will be very simple:

// Like — add a new like item to the database
func Like(w http.ResponseWriter, r *http.Request) {
    client := utils.DBClient()
    entry := CreateEntry(r, 1)
    _, err := entry.DBCreateOrUpdate(client)
    if err != nil {
        utils.SendInternalServerError(w, err)
        return
    }
    utils.SendSuccess(w)
}

We have to initialize database client and our entry (with value equals 1, for Dislike it will be a -1 value) and create or update it in database. After that, we return success or failure to the user.

Code for other functions is very similar and you can find it in files linked below.

kamilpowalowski/recommendations-go

kamilpowalowski/recommendations-go

Recommendations

For recommendation, I decided to use a library called regommend: Recommendation engine for Go. To use it I have to:

1. download all entries stored in the database (I’ll return to this in Recommendations appendix section) for the given client,

2. load it to Regommend table,

3. run recommendation engine,

4. map and return a result to the user.

All these steps you can find in recommendations.go file that you should also create in users directory.

kamilpowalowski/recommendations-go

Recommendation appendix

Loading all client entries from the database each time /recommendations is called is definitely not an optimal solution. It will work for small projects, but for bigger services it must be changed. Regommend can store all data and offer additional functions that will update the state. But we can’t use it due to the serverless nature of our solution. Since the purpose of my experiment was to check another serverless solution I decided that I won’t be looking for the solution to this problem.

Project wrap up

That was the last step in project implementation. You can run and test your project using now dev command and test your API using Postman, Paw or any other tool that you prefer. If you are ready, you can deploy it to Now with now command. For your convenience, the whole projects’ code and small Postman collection to test it are available on GitHub repository.

In the Summary section, I’ll discuss what I’ve learned and what conclusions do I have after working on this experiment.

Summary

I hope that after looking at the size of this article, you have an idea that this “small” project is not as small as it seems to be. I think the main reason for this is a distributed environment of our serverless architecture. I used two different services (Now and FaunaDB), and both of them were new to me. Both required some cognitive work to understand and make them work together. Combining with Go as a development language makes a project that should take one day a week-long challenge.

But after all, I’m happy with picked solutions:

• ZEIT Now is very easy to use. Good documentation, simple CLI and clear design make it accessible even for someone who spend his days writing mobile applications, and his only contact with the backend are usually related to the Firebase Firestore or REST API.
• FaunaDB despite its NoSQL nature is more intuitive than Firestore at the begging. I liked the comparisons they made to the relational database. There is one additional advantage which was announced just a few days ago — FaunaDB now supports GraphQL FaunaDB now support GraphQL.
• Go turns out to be a suited solution for writing short serverless functions. Still, I probably spend most of the time figuring things related to Go like: how packages should work, how to use FaunaDB driver or how to deploy and build everything on Now. But I don’t consider this as a problem with Go. It was a learning curve that I have to take.

Would I recommend tools presented above for you? I couldn’t do that. This decision is only on your shoulders. But I already know that I’ll use Go, Now and FaunaDB for my next side-project.

At 10Clouds, we develop outstanding apps. Check out our services website to get more details or just contact us. From business consulting, through choosing the right technology, to delivering perfect code, our team will make the best use of its wide skills to make your product successful.