AI vs ML, buzzwords or the future?
I first found myself needing an AI/ML solution when the business I was in was being systematically hit by fraudulent orders. Ecommerce is no stranger to fraud, but each business faces it in different and unique ways. We were being targeted from many locations, multiple payment types, every product and any delivery location. There was a pattern and behaviour that a human could spot after the event but we needed to be able to spot it before the order was shipped.
We were a small engineering team without the enterprise budget to outsource this problem, as was common of the day, so we decided to build a solution. Importantly we had to do so before our next peak buying period when hundreds of thousands of pounds were at stake.
For context, each order that was shipped and later was found to be fraudulent cost us the value of the product, the shipping cost, the man power to make and pack the product then, the real kick in the teeth we’d have to refund the total order cost and be charged a fee (c £15) for the fraud to be investigated by the payment gateway. So we needed to solve this!
Placing an order online follows a specific set of tasks, a clearly defined route from the product page to the payment page. Each field requires defined data to be inputed and validated and ultimately approved by the payment provider. Our challenge was to spot orders that were fraudulent, despite the payment provider approving them. We only had a handful of weeks to solve this but had a wealth of data to draw from, plus that human experience of what to look for.
Today, you might assume that AI or ML are the go to tasks for such problems and they really are. But two things were against us, neither were off the shelf at the time and both required a model to be trained, time we didn’t have. So we built our own model based on our past data, common behavioural patters powered by our own experience. Human intelligence rather than a truly artificial one.
We built our own model based on common data patterns that provided a weighted score which allowed our fraud team to evaluate the order before it was shipped. The model was rather binary, each flagged attribute would escalate the order to another attribute and add to the weight of the score. This is simple pattern detection and in fact precedes the concept of AI or ML. Mathematically our execution was closer to the Markov Chain principle. Whilst we had no need for stochastic modelling, we did want the predictive outcome such modelling provided. The net result was a success and we saved the business a significant amount of money. Many years later, the model is still in place today and each day the data that feeds it helps the model become better at identifying the next fraudulent order.
That’s not Machine Learning, but the basic principles are close. Importantly our solution was binary, the model was static and was only as powerful as the available data that fed it. True machine learning learns from each data point and becomes stronger over time. This is interesting as, much like parenting you can teach the model, it gets better over time but you can steer it back on course when it learns bad habits. One big benefit of the binary model we built was that it was not subject to bias, it provided a score and a human made a decision based on that score. But the score didn’t learn to treat Visa differently to Master Card over time as AI or ML could.
A model build in ML will learn from the data it is fed and over time become more aware. By design, an ML model can be taught to be as simple or as powerful as required. Importantly the factors that power the model are transparent.
That last point is what interests me most about the distinction between AI and ML. Artificial Intelligence, has recently become the buzzword of the meeting room. Blockchain was the word of choice previously and seems to have had its day. Don’t take my word for it, this is what Google Trends has to say.
It’s fair to say that Blockchain is new and AI was first concived in 1956 but the search spike speaks volumes.
Today, I’m often in meetings when someone finds a way to include AI. I’m left wondering do you mean AI or ML? I work in pensions, other people’s money and finding a way to provide good outcomes. So when we hear buzzwords we are taught to tread carefully. The pension industry is rightfully regulated so you tend to stick to your guns. A good SMPI calculation (deterministic modelling) which aims to predict your money in retirement based on your pension value, earnings, retirement age and a number of assumptions around spending. A static calculation that is closer to the fraud model described earlier. If ML was applied here it could take the same model but produce multiple outcomes based on changes to the assumptions and lifespan. This stochastic model is familar in finance, the idea that a user can select from a heat map of potential outcomes that represent many potential scenarios. Unlike the SMPI calculation, it’s highly probable that one of the stochastic projections provided via an ML model would be accurate to the end user. The issue here is educating the end user to understand the various outcomes means you stray toward advice, which is frowned upon in the industry.
But what about AI? Unlike that SMPI calculation or in fact ML, you cannot inspect the meta data that created the answer in an AI model. By design, AI is a black box, you feed it data and it learns and evolves with each new interaction, but it doesn’t provide a receipt for each transaction, you have to trust its decision. This is where the term machine bias is founded. Unlike ML and others, you cannot review the meta data that made the decision in an AI model and then correct it if you feel it is off course. To a degree this is accepted in gaming or gambling you want the results to vary and you may want to skew the results in a particular direction. But if the model develops a bias you have to start again or risk losing customers.
Last year Amazon found themselves in an AI media storm around this very topic. Like any big firm they have a big recruitment process and wanted to see if they could streamline it. The idea was simple, use historic CVs and job descriptions and compare successful and failed candidates against new CVs for similar roles. This is totally logical, scan the CVs for attributes that matched successful candidates in the past and help the pre-screening process meaning you only have to vet the candidates that are likely to get to the next phase, what could go wrong?
The problem was a PR one first, an ugly truth was exposed, the AI, based on the data provided had developed a bias that reflected a male orientated recruitment process. This wasn’t just a reflection of the text in a CV but importantly the outcome that Amazon (and the rest of the tech industry) tended to hire more men than women. The model was developed over many years but in a period where we are actively trying to encourage more women into technology, this bias, artificial or not, was not acceptable.
In an office situation, the CEO could have a chat with the head of HR and put an end to this. But with AI you cannot have that conversation. AI has to learn by itself and to do that you have to lock it down and let nature take its course. Yes, you could feed it false information, a batch of female CVs that you say were successfully hired but in doing so you are cheating the results and may as well do the leg work yourself. The only answer was to shut down the programme.
The ultimate irony being that the Amazon example would also reject the ideal male candidate who stated that he coached a female basket ball team. The bias was unforgiving.
So when I hear people mention the idea of using artificial intelligence to predict ones wealth in retirement I am both excited and scared in equal measure. My instinct is to embrace the new, let’s build that AI model, but I’d be privately thinking let’s also build an ML model at the same time as a back up and compare notes.
Whilst I am keen not to create a bias with this account, the below is the Google Trends report between AI, ML and Blockchain. Read from that what you will…
