How to use Machine Learning to solve Personalized Recommendations in Go

For example, we can use machine learning to automatically find the right people to tell about our tech conferences

Personalization and recommendations are great for everybody—they can cut down the noise and take users right to the stories, articles, or products that they are most interested in.

Building a recommendation engine from scratch is a daunting task—most wouldn’t even know where to start—but I wanted to show how easy this problem was to solve using Suggestionbox.

Suggestionbox is a personalization and recommendation machine learning engine in a Docker container — read more about it in this introductory post.

How will it work?

We will create a model in Suggestionbox where each choice is a different conference. We’ll then ask Suggestionbox to predict which conferences we should show to the user, based on a few things we know about them. If one of the conferences catches the user’s eye, we’ll reward the model in Suggestionbox so it can learn from that.

Start with the data

Let’s start by defining some data structures to model conferences and users:

// Conference represents a single conference.
type Conference struct {
  Title    string
  Tags     []string
  City     string
  URL      string
}
// User is a human.
type User struct {
  Age       int
  Country   string
  Interests []string
}

The interesting thing to note here is that there are no shared fields. For example, we are not blatantly asking the user to pick conference tags which we can then search on—instead, we are going to let machine learning discover any patterns for us. And while the Conference has a City, a User has a Country—again, we won’t bother with any geocoding or anything here, we’ll just trust Suggestionbox to figure out what works, automatically.

Some test data

Next, we are going to hard-code some upcoming conferences—in the real world, you’ll probably load these from a database or something.

var conferences = map[string]Conference{
  "Gophercon": {
    Title:    "Gophercon",
    Tags:     []string{"golang", "paid", "talks"},
    City:     "Denver",
    URL:      "https://www.gophercon.com/",
  },
  "FOSDEM": {
    Title:    "FOSDEM",
    Tags:     []string{"opensource", "python", "hackers", "beer", "devrooms", "free", "talks"},
    City:     "Brussels",
    URL:      "https://fosdem.org/2018/",
  },
  "Golang UK Conference": {
    Title:    "Golang UK Conference",
    Tags:     []string{"golang", "paid", "talks"},
    City:     "London",
    URL:      "https://www.golanguk.com/",
  },
  "PHP UK Conference": {
    Title:    "PHP UK Conference",
    Tags:     []string{"php", "opensource", "free", "talks"},
    City:     "London",
    URL:      "https://www.phpconference.co.uk/",
  },
  "KubeCon": {
    Title:    "KubeCon",
    Tags:     []string{"kubernetes", "devops", "free", "talks"},
    City:     "Seattle",
    URL:      "https://kubecon.io/",
  },
}

This is just a map of conferences where the key matches the title, which will help us look them up by ID later.

Simple Go app

We are going to build a simple web app in Go and use the Machine Box Go SDK to interact with Suggestionbox.

Let’s add the plumbing that all Go programs expect:

var sb *suggestionbox.Client
func main() {
  var (
    sbAddr = flag.String("suggestionbox", "http://localhost:8080", "Suggestionbox endpoint")
  )
  flag.Parse()
  sb = suggestionbox.New(*sbAddr)
  http.HandleFunc("/create-model", handleCreateModel) 
  http.HandleFunc("/", handleHomepage)
  http.HandleFunc("/clickthrough", handleClickthrough)
  if err := http.ListenAndServe(":8081", nil); err != nil {
    log.Fatalln(err)
  }
}

Here, we declare and initialize a suggestionbox.Client called sb, and setup our handlers using http.HandleFunc.

There are three endpoints here which map to the three main operations in Suggestionbox:

• handleCreateModel—This administrative endpoint will create the model in Suggestionbox
• handleHomepage—The homepage is where we will display the predicted choices
• handleClickthrough—When the user clicks on a conference, we’ll send them to the conference URL, but not before we reward the model so that Suggestionbox can learn

Suggestionbox choices

We need to translate our Conference type into a suggestionbox.Choice, which we can do with a simple method:

// Choice makes a suggestionbox.Choice for this
// conference.
func (c Conference) Choice() suggestionbox.Choice {
  return suggestionbox.NewChoice(c.Title,
    suggestionbox.FeatureText("title", c.Title),
    suggestionbox.FeatureKeyword("city", c.City),
    suggestionbox.FeatureList("tags", c.Tags...),
  )
}

Using the helpers from the SDK, we are essentially creating a Choice whose ID is the title of the conference. Each Choice is made up of three features (or properties): the title, the city and the conference tags.

Notice that each feature is a different type; this is because the data will be treated slightly differently by Suggestionbox.

Recommended conferences

Since Suggestionbox will return a reward ID for each predicted choice, we need to keep track of that. Let’s do that in a new type:

// RecommendedConference is a recommended conference.
type RecommendedConference struct {
  Conference
  RewardID string 
}

All we are doing here is embedding the Conference type into a new struct and adding a RewardID string.

Create the model

Now we are ready to implement our first handler where we will create the model.

func handleCreateModel(w http.ResponseWriter, r *http.Request) {
  model := suggestionbox.NewModel("model1", "Conferences")
  for _, conf := range conferences {
    model.Choices = append(model.Choices, conf.Choice())
  }
  model, err := sb.CreateModel(r.Context(), model)
  if err != nil {
    http.Error(w, err.Error(), http.StatusInternalServerError)
    return
  }
}

Yes really, this will create a machine learning model that is capable of making personalized recommendations from our conferences.

We set the model’s ID to model1 and its name to Conferences before iterating over all the Conference items and appending its associated Choice to the Model.

Making predictions

Our homepage handler is where we will ask Suggestionbox to make some predictions based on user characteristics:

func handleHomepage(w http.ResponseWriter, r *http.Request) {
  ctx := r.Context()
  user := currentUser(ctx)
  predictRequest := suggestionbox.PredictRequest{
    Inputs: []suggestionbox.Feature{
      suggestionbox.FeatureNumber("age", float64(user.Age)),
      suggestionbox.FeatureKeyword("country", user.Country),
      suggestionbox.FeatureList("interests", user.Interests...),
    },
  }
  ctx, cancel := context.WithTimeout(ctx, 500*time.Millisecond)
  defer cancel()
  prediction, err := sb.Predict(ctx, "model1", predictRequest)
  if err == context.DeadlineExceeded {
    // TODO: fallback to defaults - Suggestionbox didn't reply in time
  } else if err != nil {
    http.Error(w, err.Error(), http.StatusInternalServerError)
    return
  }
  var recommendedCons []RecommendedConference
  for _, choice := range prediction.Choices {
    recommendedCon := RecommendedConference{
      RewardID:   choice.RewardID,
      Conference: conferences[choice.ID],
    }
    recommendedCons = append(recommendedCons, recommendedCon)
  }
  renderPage("homepage", recommendedCons)
}

We get the User and make a suggestionbox.PredictRequest by adding Feature items (like we did when we created Choice objects earlier).

The renderPage method is out of scope for this post — but you can imagine it renders a template with the conferences listed.

In this case, we are adding the age as a number, their country as a keyword and their interests as a list. We aren’t telling Suggestionbox what to do with this data, it will figure that out for itself!

Next we create a context.Context with a timeout (500ms)— if Suggestionbox takes too long, we can’t wait around because users are waiting for the page to be rendered.

Calling sb.Predict will do the work of predicting which of the choices (conferences) this user is likely to engage with.

All being well, we turn the returned Choices into RecommendedConference types being sure to set the right Conference and RewardID.

The RewardID is a special string that we will use to tell Suggestionbox that it was correct if the user clicks on the recommendation.

Rewarding the model

The final handler we need to implement is the one that gets called when the user selects a conference. We will reward the model (since it was correct, the user did indeed engage with the recommendation) and redirect the user to the URL of the conference.

From the user’s perspective, this will be seamless — it’s always better to implement this kind of technology in a passive way without slowing users down.

We expect this endpoint to be called with the following URL parameters:

/clickthrough?conferenceID=Gophercon&rewardID=bef78678dadad17da8d1d8

Here’s the code:

func handleClickthrough(w http.ResponseWriter, r *http.Request) {
  q := r.URL.Query()
  conference := conferences[q.Get("conferenceID")]
  http.Redirect(w, r, conference.URL, http.StatusFound)
  reward := suggestionbox.Reward{
    RewardID: q.Get("rewardID"),
    Value:    1,
  }
  if err := sb.Reward(r.Context(), "model1", reward); err != nil {
    log.Println("reward failed:", err)
  }
}

We get the conference, and redirect the user immediately — so they can be on their way.

Then we create a suggestionbox.Reward using the RewardID and a Value of 1.

Most of the time the Value will be 1, but you can use reward values to add weight to rewards in a positive or negative way—but that’s for more advanced use cases.

Calling sb.Reward will tell Suggestionbox that it correctly predicted a conference the user was interested in.

Conclusion

Without much code at all, we’ve just implemented a state-of-the-art recommendations engine for our conferences.

Suggestionbox will keep evolving, experimenting, and guessing until it starts to notice patterns in the rewards it receives, at which point it will start to exploit that learning.

Over time, as more people use the model, it will learn about which features of a user draw them to various features of conferences — all learned passively from real user activity.

For example, it’s entirely possible that Suggestionbox would automatically start to:

• Suggest nearby conferences by noticing that countries and cities are important to users
• Notice a pattern between interests and the conference tags (perhaps “devops” and “docker” get linked)
• Discover that junior developers (younger people) might be more interested in free conferences, where senior developers might care more about other things

The point is, you don’t need to do all this thinking/guessing, you can let Suggestionbox do it for you.

We’d love to hear about what you build with this technology, and how it can be applied to your particular user cases.

You can play with Suggestionbox yourself by opening a terminal and typing:

docker run -p 8080:8080 -e “MB_KEY=$MB_KEY” machinebox/suggestionbox

If you need an MB_KEY, get one for free from the Machine Box website.

What is Machine Box?

Machine Box puts state of the art machine learning capabilities into Docker containers so developers like you can easily incorporate natural language processing, facial detection, object recognition, etc. into your own apps very quickly.

The boxes are built for scale, so when your app really takes off just add more boxes horizontally, to infinity and beyond. Oh, and it’s way cheaper than any of the cloud services (and they might be better)… and your data doesn’t leave your infrastructure.

Have a play and let us know what you think.