Getting Spark, Python, and Jupyter Notebook running on Amazon EC2
Jose Marcial Portilla
417

One useful update you may add is that pyspark is already compatible with python 3.6 and it could be installed directly through pip instead of downloading it manually and add to the PATH.

On the “limited resources” machine that we get from AWS, I get a memory error after running pip install pyspark. There is an answer to this problem https://stackoverflow.com/a/31526029 which recommends to use a flag and this worked for me:

pip — no-cache-dir install pyspark

Show your support

Clapping shows how much you appreciated Federico Pereira’s story.