Install Spark on Mac (PySpark)
Michael Galarnyk
27412

Hello,

I have followed the steps , but I am getting following error:

Error in pyspark startup:

IPYTHON and IPYTHON_OPTS are removed in Spark 2.0+. Remove these from the environment and set PYSPARK_DRIVER_PYTHON and PYSPARK_DRIVER_PYTHON_OPTS instead.

I have downloaded : spark-2.2.0-bin-hadoop2.7

In my ~/.bash_profile I have following:

export PATH=”/Users/spandan.chakraborty/anaconda/bin:$PATH”

export SPARK_PATH=~/spark-2.2.0-bin-hadoop2.7

export PYSPARK_DRIVER_PYTHON=’jupyter’

export PYSPARK_DRIVER_PYTHON_OPTS=’notebook’

alias snote=’$SPARK_PATH/bin/pyspark — master local[2]’

Pls advise

Thanks,

Spandan

Like what you read? Give spandan chakraborty a round of applause.

From a quick cheer to a standing ovation, clap to show how much you enjoyed this story.