pysparkling is a native Python implementation of PySpark. Stream processing is considered to be one of the most important features of Spark. PySpark provides a Python interface to Spark’s StreamingContext and supports consuming from updating HDFS folders and TCP sockets…

Originally published on September 22, 2016.

Example that processes a text corpus with word2vec in Spark on Databricks.

Introducing parallel processing in pysparkling and benchmarking…

Collection of links to talks.