AB testing based on facts.

Decision engine

Michał Lebida
AzimoLabs
Published in
3 min readJul 16, 2018

--

We’re always looking for ways to surprise and delight the users of Azimo, our international money transfer service. That’s why we created a a special “decision engine” that automatically rewards customers depending on a set of criteria, such as their personal data or their behaviour on our website and apps.

This article gives a brief overview of the decision engine’s code and how it works.

How does it work?

We wanted to create a simple decision-making model that everyone in our team could understand. We settled on testing a data model of:

  • Fact: Something we know about the user
  • Operator: Should it be equal or greater than a given value?
  • Value: The value that the test should be run against

With these assumptions we created an engine for decision-based services.

Implementation

Our engine is written in Scala, with the addition of Shapeless library.

Before we look at some example code, we need to get some basic understanding of generic programming and ADT.

What are ADTs (Algebraic Data Types)? “Shapeless guide” says:

Algebraic data types (ADTs) are a functional programming concept with a fancy
name but a very simple meaning. They are an idiomatic way of representing
data using “ands” and “ors”.

The code below shows ADTs differentiating between bank account types:

Account is SavingsAccount OR CheckingAccount, and SavingsAccount has amount AND forHowLong. In more formal terminology:

  • ‘or’ types (Account) are called Coproducts.
  • ‘and’ types (SavingsAccount and CheckingAccount) are called Products,

We use ADTs because they are safeguarded by Scala Exhaustiveness Checking. For example, let’s say we write the code below:

Scala compile would warn us of a missing implementation:

match may not be exhaustive.
It would fail on the following input: BussinesAccount(_)

Let’s now look at some example code.

Building a model

Let’s start with defining some random facts about our users, such as whether they like ice cream 😋:

And some available operators, such as greater than, lower than etc:

And some available values:

Operator “:+:” is syntactic sugar for creating generalized Either with arbitrary number of available options. It means that ValueType can be either one of defined types. For more information about shalepess Coproducts visit project website.

Now that we have a model, we can define a test case:

With all definitions ready we can implement the decision engine in the following way:

Now we can define test cases which can be run against some facts. The code below shows data about our users (whether they like ice cream, how many times they logged in last month etc.) and separates them into buckets depending on which criteria they do or don’t satisfy.

The results

By combining data about our users with our decision engine, we can now automatically reward our users based on set criteria. The simplest use case might be to automatically give someone a voucher on their birthday.

More complex examples might take into account how many times a user has logged in, how many transfers they have made using Azimo, what kind of payment method they used etc. Given that we have thousands of data points about our users and their behaviour in our network, the possibilities are almost endless.

You can find an example of the project on Github.

Towards financial services available to all

We’re working throughout the company to create faster, cheaper, and more available financial services all over the world, and here are some of the techniques that we’re utilizing. There’s still a long way ahead of us, and if you’d like to be part of that journey, check out our careers page.

--

--