CONTINUOUS AI: THINK TWICE, YOU HAVE TO MARRY WITH YOUR AI PROJECTS
In our previous blog post, we mentioned about the possible reasons for the AI project failures after the production. Check our, Drifting Effects on AI Models: Why Continuous AI is a Must? post for the detailed explanation of the changes in model lifecycle. In this post, we will continue about the systematics to approach continuous model development requirements.
Artificial intelligence technologies are boosting our decision-making process power in various topics from selecting a movie in a streaming service to calculate the risk scores of a customer in a bank. But have you ever thought about the average time of an AI project? Or as a data scientist have you ever been asked to estimate closure time? Nowadays, most companies aiming for solving the problems by leveraging AI capabilities. Yet, companies are expected to execute 35 AI projects on average in 2022 _¹ . But, what about the continuation of these models? It is good to look what will be the future look like with these one-time developed model processes. Even today, 70% of the companies has reported that their AI projects has no impact or very minimal impact in long term on their operations_².
Continuous Care is Essential for the Quality
Over the years, projects are turned to time limited enterprise to achieve pre-determined goal by definition. Especially in AI projects, develop once run anytime concept is far from realistic needs and requirements of an algorithm. So, the answer to the question in the beginning may scare you, but lifelong developments and adaptions are the key of an AI project to sustain its effectiveness. In other words, there should be a continuous bonding with you and your projects especially after deployment to the production environment.
When an AI model runs in real life, there will be inevitable changes in data, concept, or business processes in countless scenarios_³. So, being in control to plan proactive interventions (retraining etc.) is critical. As the environment is constantly changing, continuous care is not an option but an obligation.
One Size Fits All Structures are not Working
AI project’s lifecycle can be related to human lifecycle. Although there are pre-determined steps along the journey of a person such as go to elementary school, then go to middle school, each person experiences different challenges in this path. Likewise, in AI models after the production, the problems (thus the solutions) are mostly non-scalable, and there are infinite number of possibilities in the way. Each AI project must be unique when it comes to deciding the improvement actions. So, it is hard to implement a continuous AI model with a fully established method with an action plan.
Plan Early for Reinforced Success
One of the key factors for the achieving ultimate success following production is to be aware of the continuous care that will be needed, ahead of the project and planning the subsequent tasks accordingly. This includes setting a project documentation structure and record model development milestones, assumptions and exceptions in detail.
Considering of the continuous model development; standard model development process becomes an iterative journey rather than a straight process. Since initial steps of the model development process (business understanding, data understanding etc.) may be revisited in case of a reduction in model quality (because of data drifts or concept drifts_³) in Continuous AI approach.
You Cannot Leave Your AI Project Alone
In a nutshell, once you develop an AI project, deteriorations are inevitable over the time and some of these flaws cannot be solved with automatic version changes, bugfixes or failure-handling mechanisms. Deep-dive analysis and specialized solution (such as, covid effect, business process changes) may be required. Hence, you should always keep an eye on AI project for proactive support and quality certainty.
[2]: https://research.aimultiple.com/ai-fail/
[3]: https://www.explorium.ai/blog/understanding-and-handling-data-and-concept-drift/
