Open in app
Sign inGet started
Tagged in

Spark Connect

Towards Data Engineering
Towards Data Engineering
Dive into data engineering with top Medium articles on big data, cloud, automation, and DevOps. Follow us for curated insights and contribute your expertise. Join our thriving community of professionals and enthusiasts shaping the future of data-driven solutions.
More information
Followers
2.1K
Elsewhere
More, on Medium
Go to the profile of Sebastian Daberdaku
Sebastian Daberdaku in Towards Data Engineering

Performing Delta Table operations in PySpark with Spark Connect

Introduced with Spark 3.4, Spark Connect provides a decoupled client-server architecture allowing remote connectivity to Spark clusters using the DataFrame API. It is a very neat feature that allows users to run PySpark…

Read more…