Setting up SPARK 3.1.1 Livy and Minio object storage

A guide for modern data engineer to invoke spark program using livy and Minio object storage using docker images and minio command line client

Most of the companies today have cloud presence either in the form of public cloud( like Aws, Azure or GCP) or private cloud. The next phase of the transition, in my opinion, will be towards hybrid cloud. Hybrid cloud can be mix of public cloud( for example Aws and Azure ) or public — private cloud(for…