Running PySpark on EKS Fargate: Part 1

It will be a series of three blogs where we will learn how to run PySpark jobs on AWS EKS Fargate.

Running PySpark on EKS Fargate

The contents will be spread over the blogs as follows:

  1. Part 1: Prerequisites and Playing with Docker Image
  2. Part 2: Manually Merging one version of Spark with that of Hadoop
  3. Part 3: Running the actual spark Job.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store