Aug 23, 2017 · 1 min read
spark on windows and integration with jupyter notebook is not so difficult
1) install java
2) install 7zip
3) download spark tgz file
4) unzip using 7zip
5) create an ENV variable SPARK_HOME with —-> file_location_of_unzipped_spark_folder/bin
6) add this varaible to path
7) create an ENV varaible PYSPARK_DRIVER_PYTHON with —> jupyter
8) create an ENV variable PYSPARK_DRIVER_PYTHON_OPTS with —> notebook
9) run in cmd line → pyspark --master local[2] with 2 local nodes