This article is for Data Scientists and ML Engineers who are running Spark jobs using EMR cluster on AWS and have faced challenges…