A Manifest for Continuous AI
When a company decides to invest in an Artificial Intelligence/ Machine Learning solution to solve important problems, their primary focus will be, of course, solving the problem. So, the marathon begins with collecting and cleaning data, training models, analyzing metrics… etc. In essence, all the steps required for a successful model development. Then comes the demo, testing, prototyping phase, and visual reports for executives. After these processes are done, we are ready for production. Theoretically, it should be easy from now on. After all the hassle of development is over, all the metrics, reports and outputs are as they should be, we can commence the last step Deployment. It is not easy as it seems, AI projects are in need of continuous care. For further elaboration on this subject, check out our blog post Think Twice, You Have to Marry With Your AI Projects.
Post-Deployment is somewhat like a magician’s hat, it looks like a posh top-hat, but a rabbit can jump out of it in any minute, or multiple rabbits carrying matchboxes! One moment all is fine, in another moment everything is on fire, a magician is nowhere to be found and you are trying to combat the flames while one of your colleagues is trying to put the rabbits back into the top-hat.
In 2020, Deeplearning.ai reported that “only 22 percent of companies using machine learning have successfully deployed a model”_¹ . To improve this situation, we embraced the Continuous AI approach. During our quest for a healthy AI project, several rules started to form. We gathered these rules, in a manner resembling a manifest, under five main topics.
a) Proactive
An AI Lifecyle is prone to problems. Besides the operational problems, changes like data drift and concept drift are also some things to consider. Dealing with complications like that after encountering them is a way, sure, but it is highly inefficient. Hence, our approach here is based on following our models closely, so that we can foresee what model will need and take the necessary precautions. We give monitoring systems the utmost importance because they are vital to our proactive approach.
Without an efficient monitoring and notification system, it won’t be possible to respond to events that can affect accuracy and performance on time. Therefore, monitoring systems need to be ready before production and performance tracking alerts should be set. Also, a central monitoring system is a necessity while handling multiple projects as it enables us to track multiple projects productively.
b) Continuous Improvement
Our world changes around us. With that progression, today’s requirements may not match tomorrow’s needs. When defining a continuous improvement process, it is important to include the customers’ requests. In this constantly evolving environment, the customer expectation keeps changing, so it is important to listen to the customers’ feedback for improvements_². We merge our customers’ feedback, with our insights, so that we can continue to improve our models and to create value for customers.
c) Specialized Team
Maintaining a continuous life cycle is a team effort. We embrace the agile working standards for our structure. Every project has its assigned small team of skillful individuals with different areas of expertise. These areas of expertise include cloud platforms (Azure, AWS, Google Cloud), classical on-premise development tools (SQL, Python) and monitoring tools (Airflow, Prometheus and Grafana).
When an incident occurs, this team is the first to respond and capable to address almost all problems. If an issue that cannot be addressed by this team occurs, they are the ones who will escalate it to the project developers.
d) Holistic
System design and continuous monitoring are vital aspects of a healthy AI Project. But the real growth factor is understanding the business and creating value for our customers. Bearing this in mind, we approach the model as a whole. We give both System Design and Continuous Improvement equal attention. As a result of this, every aspect of the process has ample production time to achieve the best possible outcome.
e) Standardized
Our post-development processes are highly standardized. When we are taking over the project, we proceed according to our checklists. By considering the platforms on which the project was developed, we tick our boxes one-by-one, ensuring that nothing is left out and everything is in order. So that we are ready to sustain an effective maintenance process.
Just when you think you are prepared for everything that your model can throw at you, it is still possible for your model to become outdated in the face of improving technology and evolving customer expectations. While following the steps of our manifest, it is our responsibility to always keep our senses open to the wind of change.
