The Cold Start Problem with AI
If you have become a Data Scientist in the last three or four years, and you haven’t experienced the 1990’s or the 2000’s or even a large part of the 2010’s in the workforce, it is sometimes hard to imagine, how much things have changed. Nowadays we use GPU-Powered Databases, to query billions of rows, whereas we used to be lucky if we were able to generate daily aggregated reports.
But as we have become accustomed to having data and business intelligence/analytics, a new problem is stopping eager Data Scientists from putting the algorithms they were using on Toy Problems, and applying them on actual real-life business problems. Other wise known as the Cold Start Problem with Artificial Intelligence. In this post, I discuss why companies struggle with implementing AI and how they can overcome it.
Any company either startup or enterprise, who wants to take advantage of AI, needs to ensure that they have actual useful data to start with. Where some companies might suffice with simple log data that is generated by their application or website, a company that wants to be able to use AI to enhance their business/products/services, should ensure that the data that they are collecting is the right type of data. Dependent on the industry and business you are in, the right type of data can be log data, transactional data, either numerical or categorical, it is up to the person working with the data to decide what that needs to be.
Besides collecting the right data, another big step is ensuring that the data that you work with is correct. Meaning that the data is an actual representative of what happened. If I want a count of all the Payment Transactions, I need to know what is the definition of a Transaction, is it an Initiated Transaction or a Processed Transaction? Once I have answered that question and ensured that the organization agrees on it, can I use it to work with.
With the wide adoption of SCRUM and frequent releases, companies have to devote resources to ensure that the data is correct. Companies could, add new sources of data, changes in the code that can have an impact on the logged data or even outside influences like GDPR or PSD2, that can cause the data to be altered because it needs to be more secured or stored in a different way. By ensuring that during each process the correctness of the data is ensured, only then can you move on to the next phase of analytics.
Even though AI is currently what everybody talks about, before we get there we still have to take an intermediate step, which is Analytics. What I mean by Analytics, is the systematic computational analysis of data or statistics. In most companies, the process to get to the visualization of the data might be known to few, but the impact it has on each department is tremendous.
Companies need to determine which Key Performance Indicator’s (KPI’s) actually drive the business. Working in the Payments Industry, my KPI’s include Processed Revenue, Transaction Costs, Profits, Authorization Rates, Chargeback Rates, Fraud and many others that provide me with the information to manage the performance of the business. For a Taxi App, KPI’s might include, Revenue, Profit, Average Pick-up Time, Average Ride Time, Active Users and Active Drivers.
From those KPI’s, a company can then decide what type of reporting or dashboards are necessary for the business users to make informed decisions and work on automating the systematic computational analysis of the data or statistics.
But as the volume of data increases from KB’s to TB’s, and business users are looking more and more at aggregated reports and visualizations of the data, the chances of detecting smaller issues reduces significantly. It is only then, that implementing AI can become a worthwhile investment of time and resources.
Having determined the KPI’s that help steer the business, Artificial Intelligence can be used to improve these KPI’s.
Posted on 7wData.be.