Towards Data Engineering
Published in

Towards Data Engineering

Aman Ranjan Verma

Apr 13, 2020

3 min read

Running PySpark on EKS Fargate: Part 1

It will be a series of three blogs where we will learn how to run PySpark jobs on AWS EKS Fargate.

Running PySpark on EKS Fargate

The contents will be spread over the blogs as follows:

  1. Part 1: Prerequisites and Playing with Docker Image
  2. Part 2: Manually Merging one version of Spark with that of Hadoop
  3. Part 3: Running the actual spark Job.

--

--

--

More from Towards Data Engineering

The publication aims at extracting, transforming and loading the best medium blogs on data engineering, big data, cloud services, automation, and dev-ops.

Read more from Towards Data Engineering

Recommended from Medium

Heathrileyo

Giving Our Enemies an Avoidable Homing Attack

Bontengkeripik

➤鬼灭之刃 剧场版 无限列车篇 完整版本 (2020-HD) Kimetsu no Yaiba: Mugen Ressha-Hen 完整版觀看電~看电影.

themeclever

Dell Poweredge 2850 Raid Controller Driver

Angie Maticorena

Sharing is caring, even in coding

squirrels sharing a celery
Auryn Engel

5 Things you should avoid while doing microservices

Al Heck

Variables — What are they?

Martin Roest

How we tackle Continuous Delivery at Ibuildings

Gav Richards

in

Aiir

New in Aiir: Simpler Programme Images

AboutHelpTermsPrivacy

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Aman Ranjan Verma

Aman Ranjan Verma

Data engineer @Flipkart, I post weekly. https://www.linkedin.com/in/ar-verma/

More from Medium

Byoung Jin Yu

in

hatiolab

Data Preprocess with AWS Glue

Durga Gadiraju

in

Data Engineering on Cloud

Databricks Certified Associate Developer — Apache Spark 3.x

Dheeraj Thasma Ravindranath

in

SquareShift

How to resolve issues with Multiple Glue RDS Connections: 1 Trick That Works

Shivam Shrivastava

in

AWS Tip

Load data into Redshift from S3

Help

Status

Writers

Blog

Careers

Privacy

Terms

About

Knowable