Online and Batch Based ML Execution from Same Python Code Preserving Pre and Post Transformation States and Affinity
Article written with help from one of my great colleagues Sivakumar Lankada. A big thanks to him.
Objective
Evolution is an important thing in the technology world. All technologies are born out of purpose. For example, search engines were created to sort through the massive amounts of data online. With each new upgrade, technology compounds existing technologies to create something better than what was previously used. And on and on it goes.
However due to the big data evolution, there are lot of scalable architecture and platforms that are evolved quickly. There is nothing wrong with that, but the maturity/experience of a product that has been evolved over a period of many decades can never be ignored while we embrace new things and amazing innovations. It’s like saying experience cannot be compromised — regardless of whether humans or technology. Those with years of tech experience in financial sectors, please ask yourselves a question, whether the mainframe processing could be replaced by any technology. Similarly in every big data related talks, I advocate not to replace your RDBMS with big data — as this question always comes from audience. It’s not possible. RDBMS is meant for a unique…
