Parallel Inferencing using a Fast.ai model and Azure ML (ParallelRunStep)

Author: Fatos Ismali

Businesses of all sizes and shapes are embarking on Machine Learning initiatives and projects and embracing the benefits that this new technology is bringing about to their particular industry. The digitization of physical data sources that were deemed out of reach of technology, has created a deluge of data that keeps growing exponentially. This growth in data has inadvertently created a lot of technical and process challenges that are making it harder for companies to adopt Machine Learning effectively. One of the main challenges in the Machine Learning world today is scalability. Even though we like to boast about the computing power that exists today, we still seem to struggle in making effective use of it for machine learning applications. Part of the reason behind this is that training machine learning models in a distributed fashion or inferencing on new data in a distributed way is inherently complex, and programming under such paradigms is something only a few seasoned engineers can do. It is not straightforward to build a distributed training or inferencing pipeline from scratch. Many of the programming languages and frameworks out there have been build to abstract some of the concepts around distributed computing and make it easier for developers to build distributed applications. One such framework is the CUDA rogramming interface developed by Nvidia. OpenMPI is another open-source project for message passing between applications across networks with…