It all started in 2006! at eBay..
A machine learning scientist: Josh Menke , convinced me (Harshal) that Neural Networks were a better solution than logistic regression models and thousands of business rules, to solve Fraud Detection at eBay scale! We were then joined by Brian Carnes to build eBay’s first AI system: Radar.
I prepared business case and identified the right use case (more on that later), while Brian and Josh used FANN and Ab Initio to build and launch the system.
It was a massive success..
In 2009, I went to PayPal to build a similar system. But at Internet scale, meaning solving fraud detection across all websites that use PayPal. Not just eBay.
That required different scale and different technologies.
I first discovered Hadoop. And then found two amazing engineers who joined me: Evan Huang and Grahame Jastrebski! without realizing it, we built Lambda Architecture.
I remember the first prototype we built. I had written some rudimentary Map Reduce code on Hadoop, running on a retired Solaris machine. Grahame and Evan build a cluster using our windows desktops and adding hard disk purchased from local Fry’s electronics!
We called the system: vBase. For Variable Base. Similar to a Database. Our logic was Database stores data. While Variable base will store variable of features. That were pre computed (batch) and computed in real time (leveraging a cache- thus Lambda architecture).
Once again followed convincing business and tech leaders. Finding use case that makes sense. Securing funding. Preparing and executing on a ‘project plan’ for an AI project! Defining KPIs (OKRs). And measuring and demonstrating results against all scrutiny.
After a lot of blood sweat and tears, entire PayPal finally embraces that system and build next generation platform for Fraud Detection that enabled AI natively.
This is vBase! Our work on AI, deep learning, Big Data, Lambda architecture, before those terms were coined / were popular.
I am hoping to share some of those learnings and possibly continued learnings from all of us and more, as we all took our AI passion to various other companies in the valley and beyond. So stay tuned and let me know if you want hear about something specific..
