Install Spark on Mac (PySpark)

  1. Go to the Apache Spark website (link)

Common issues: Unable to load native-hadoop library for your platform… using builtin-java classes where applicable

While the native hadoop library is supported on Linux type platforms only (The library does not to work with Cygwin or the Mac OS X platform), we have to make a workaround if it effects pyspark from working.

Concluding Remarks

Please let me know if you have any questions! I am happy to answer questions in the comments section below or on the youtube video page, or through Twitter.

--

--

--

Data Scientist https://www.linkedin.com/in/michaelgalarnyk/

Love podcasts or audiobooks? Learn on the go with our new app.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Michael Galarnyk

Michael Galarnyk

Data Scientist https://www.linkedin.com/in/michaelgalarnyk/

More from Medium

Share PySpark data via JSON (and Pandas)

A Practical Guide to PySpark

Build A Machine Learning Model with PySpark

Big Data with PySpark: Feature Engineering