Real-Time Scoring: A Points Hackathon

In November of 2017 I began working at Points, and a few months later I participated in my very first hackathon (internal within the company). The purpose of these in-house hackathons is to cultivate new ideas for our current products, or to create something totally new, but still related to the core business. In order to tell you about it I need to bring some context to this post by explaining what Points does.

Points is in the business of connecting customers to loyalty rewards services.
Now, this is very general but all that you’ll need to know is that third party
applications will use our API to connect to loyalty partners. We call this
the loyalty commerce platform (LCP) and any transaction that goes through the LCP begins with validating the member with the requested partner.

The idea for the hackathon was to leverage historical data we have on a member, in particular, we wanted to calculate additional information about the member when they log in. If a member logs in through the LCP they may be just looking around at a promotional offer, or they might be interested in buying or redeeming loyalty rewards points. So what we wanted to do was to calculate the likelihood that the member will make a ‘buy’ transaction
for their current session.

Tech Stack

There were three main components required to be created/modified in order to bring this idea to fruition:

  • Data analysis: model creation which outputs a PMML file (Predictive Model Markup Language). Our CTO Owen Tran wrote a nice article
    about how we train some of our models and generate these PMML files. You can check it out here.
  • Real-time scoring engine: a RESTful service that scores PMML models (openscoring.io).
  • LCP: a python/flask application that handles the member validation request and can make calls to the real-time scoring engine.

Execution

I won’t go into details as to how we trained our models, but rather how we use the model and connected these three components together. Everyone loves pictures, so check out the figure below, which gives a general overview of the data flow within this project.

From here we can see the newly altered lifecycle of a request to validate a member. With each successful request to validate a member, we send certain member information (features) to the scoring service.

The scoring is only done in the event of a successful member validation. After all, we cannot predict something on a member that doesn’t exist. Upon success the user of the LCP API will receive their usual response, but now the LCP has also fired off an event indicating that the scoring can begin. When this happens the scoring engine receives the created member validation and sends the necessary features to the openscoring service. The result of this is then added to the details object of the member validation record which can be retrieved via the LCP.

The model we created lives on the openscoring server as a PMML file and this service has one job: evaluate the features against the stored model. The model was fairly basic in that we only evaluate against two features, namely current balance and membership level (which could be ‘gold’, ‘platinum’, etc., represented by an integer value). In the end, a request-response looks something like this:

POST /openscoring/model/buy-decision/ HTTP/1.1
Content-type: application/json
{
‘arguments’: {
‘member_balance’: 123,
‘membership_level’: 2,
}
}
HTTP/1.1 200 OK
Content-Type: application/json
{
‘result’: {
‘probability(true)’: 0.8,
‘probability(false)’: 0.2,
}
}

From this we can see that the probability that a user with 123 points, and a membership level of 2 has an 80% chance of making a ‘buy’ transaction. I’d say that’s some pretty useful information from a pretty simple model.

Reflection

Looking back on the day and a half we had to throw this together, I think we did a pretty good job considering the tight timeline. Our system wasn’t super sophisticated, but hey, that’s the nature of a hackathon — get it working enough to prove a concept. All in all I had a great time with a great team and I’ll be looking forward to the next one!

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store