Spark Diaries
“From a tiny ‘Spark’ may burst a mighty flame” ~ Dante Alighieri
The how, the what and the why?
Me and spark have had a love-hate relationship for the past 2 years or so. It all began after my difficult break up with hive and I realized that I was better off without it. I thought I should voice my opinions and offer some relationship advice to people who are navigating their way through this arduous but some what rewarding stage of life.
On a more serious note, if you are someone who is just starting spark and have a somewhat good understanding on hive and sql, I highly recommend doing an online course which uses the databricks community edition. I did a brilliant one on edX, a while ago, which is currently archived. I am not sure if it is available if you did not archive it before. There are however some available on udemy which look promising.
The databricks community edition is a great place to just start working on something small to get used to spark and they support both scala and python notebooks.