spark on windows and integration with jupyter notebook is not so difficult

1) install java

2) install 7zip

3) download spark tgz file

4) unzip using 7zip

5) create an ENV variable SPARK_HOME with —-> file_location_of_unzipped_spark_folder/bin

6) add this varaible to path

7) create an ENV varaible PYSPARK_DRIVER_PYTHON with —> jupyter

8) create an ENV variable PYSPARK_DRIVER_PYTHON_OPTS with —> notebook

9) run in cmd line → pyspark --master local[2] with 2 local nodes

)
Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade