Photo of a sick dog taken in Chiang Mai, Thailand (Image by author)

Build a Recommender System in less than 60 minutes

A Step-By-Step Guide to Building a Recommender System using Amazon Personalize.

Bad times! Ever since the Thai government has declared a state of emergency over coronavirus — all of us were encouraged to work from home. I was assigned a challenging task to conduct a webinar on the topic of “Increasing E-Commerce Customer’s Engagement amidst Covid-19”.

So, after careful consideration and given my basic background in Machine Learning — I’ve decided to make the topic of this webinar on building a Recommender System in less than an hour using Amazon Personalize.

Truth be told, I did spend some time going through tutorials on YouTube on building a Recommender System using Python — on concepts such as Cosine Similarity. At first, it was pretty straightforward but then questions started popping up. Here’s a few of them:

• How do I deploy this algorithm? How do I serve the model? Which cloud infrastructure do I need to provision?
• How do I handle rapid changes in trends and customer’s behavior?
• How do I handle new users and new items? (the classic cold start problem)

I came across Amazon Personalize and it ticks all the right boxes.

So here’s how I did it!

What is Amazon Personalize?

Amazon Personalize is a real-time personalization and recommendation service. It’s one of those instant solutions where you just add data and with a few clicks here and there, and we get our ready-to-use recommendation engine.

If you wish to follow along — I’d recommend that you download the legendary MovieLens data which contains users and ratings, this will be our input data into Amazon Personalize

MovieLens 100K Dataset

Overview of Amazon Personalize by Shivek Sachdev (Image by author)

Step 1: Data Preparation & Loading Data

Note: In order to paint a better picture — let us use an imaginary E-Commerce website for all the 3 steps in which we are going to be using Amazon Personalize to build a Recommender System.

I had to prepare a CSV file with predefined columns with the “Who bought what?” data — this is called “Interactions”.

Example of columns required by Amazon Personalize (Image by author)

So we need USER_ID and ITEM_ID, as well as TIMESTAMP (Unix Date format).

We need to dump our CSV file into an S3 bucket and for those who are new to AWS, S3 is an Object Storage. You can dump whatever you want into it but you’ll have to pay Jeff!

So without further ado — we launch Amazon Personalize!

Search for “Personalize” in the AWS Console to launch this service (Image by author)

In this first step, we’ll start by creating a dataset group in order to import our data into Amazon Personalize.

Create dataset group in Amazon Personalize (Image by author)

Then we will be redirected to the Overview Dashboard in the Amazon Personalize console. Make sure to click on Import User-Item interaction data. We will be importing our data from S3 into the Amazon Personalize dataset group simply by just pasting the path to the CSV file.

Upload datasets: User-item interaction data (Image by author)

To import our CSV, we must create a dataset name and also provide Amazon Personalize with a JSON schema. This JSON file is a schema definition and will be used to provide all the required details before we start to build our Recommendation System.

Here’s how we can visualize the JSON schema file, hope it helps!

Schema definition (JSON) vs. CSV (Image by author)

Once this is done, we can click on “Start import” to import the CSV file in order to complete our process of creating a dataset — which wraps up Step 1!

Choose the S3 location of CSV data (Image by author)

Step 2: Create a Solution (Model Training)

Moving on to Step 2. This is where you will create a Solution and choose from a selection of pre-built Recommender Algorithms to work with (called Recipes) on Amazon Personalize.

Think of it this way:

Solution and Recipes on Amazon Personalize (Image by author)

To begin this process — we start where we left of last time, from the Overview Dashboard but this time we will click on Create Solution.

Create a solution (Image by author)

Since I have very basic knowledge in Machine Learning, I went with the option of choosing AutoML as a recipe — which means Amazon Personalize will go through our data and suggest the best recipe for us.

Create a solution on Amazon Personalize (Image by author)

HRNN which stands for Hierarchical recurrent Neural Network. This algorithm is supposed to be very powerful and can handle changes in consumer behavior — perfect for this Covid-19 era!

HRNN would be the perfect solution to model changes in behavior and recommend an item that is more suitable for the consumers/shoppers.

HRNN can model changes in the behavior of consumers/users and recommend the best choice of product (Image by author)

The magic happens here — don’t be surprise, it may take a while. It’s magic after all.

For ML Geeks, here’s the benchmark comparison of HRNN vs. the rest.

State-of-the-art performance, HRNN vs. the rest on AWS Online Tech Talks

Step 3: Launch & Test Campaign

This is our last step — we will launch a “Campaign” on Amazon Personalize and will finally get to see what items our Recommended System will present to our users/customers. Creating a Campaign is the easiest step and we start from our Overview Dashboard like always.

Create a new campaign (Image by author)

Just make sure you choose the Solution in which you have just created in Step 2 and click on Create Campaign.

Creating a Campaign on Amazon Personalize (Image by author)

Now for the moment of truth — we can test the Campaign by entering a User ID and click on Get recommendations to see a list of Item ID recommended to our user/customer. As simple as that!

Testing campaign on Amazon Personalize (Image by author)

Bonus: Model Serving in Amazon Personalize

Usually when people talk about taking a model “to production,” they usually mean performing inference, sometimes called model evaluation or prediction or serving.

Campaign inference (Image by author)

In the case of Amazon Personalize, once we have created our campaign — it will automatically create and deploy a Campaign Inference Endpoint for us. All we need to do is just call the getReccomendation API to get real-time recommendations!

Integration with Amazon Personalize Campaign ARN (Image by author)

Here’s the code sample required for calling the API — we just need a very basic understanding of AWS SDK for Python (Boto3).

Make sure you have AWS SDK for Python installed.

1. Create a “Response” dictionary
2. Call get_recommendations API, pass a userId variable to get recommendations
3. Display Recommended Items (itemId will be displayed)
4. Map itemId and merge data from “Item Name DataFrame” to display Item Names (this is optional)

import boto3
personalize = boto3.client('personalize')
response = personalize.list_recipes()
for recipe in response['recipes']:
    print(recipe)
personalizeRt = boto3.client('personalize-runtime')

Run this code block to get item recommendation by passing a user_Id (I used 2 in the example below) and make sure you call the correct campaignARN!

response = personalizeRt.get_recommendations(
campaignArn = 'arn:aws:personalize:<insert region & campaign ARN>:campaign/ratings-campaign',
userId = '2')

We have accomplished this without any knowledge of ML!

We did it! Time to celebrate! Internet Memes

Great stuff! This blog would cover only the very basic (entry-level) part of Amazon Personalize. It’s great for those with or without a background in ML but there’s more to learn. In conclusion, we managed to build a state-of-the-art Recommendation System using Amazon Personalize in less than an hour. Truth be told — I probably spent more time writing this blog than actually running this whole process on Amazon Personalize!