Start/Deploy Apache Spark application programmatically using spark launcher

Sometimes we need to start our spark application from the another scala/java application. So we can use SparkLauncher. we have an example in which we make spark application and run it with another…

Shufflling and repartitioning of RDD’s in apache spark

To write the optimize spark application you should carefully use transformation and actions, if you use wrong transformation and action will make your application slow. So when you are writing application some points should be remember…

Apache spark + cassandra: Basic steps to install and configure cassandra and use it with apache spark with example

To build an application using apache spark and cassandra you can use the datastax spark-cassandra-connector to communicate with spark…

Easiest Way to connect with Couchbase using Scala

In this blog, we have explained how easily we can connect with Couchbase, store and fetch data using scala through an example.

Tools and technologies used :

Stateful transformation on Dstream in apache spark with example of wordcount

Sometimes we have a use-case in which we need to maintain state of paired Dstream to use it in next Dstream . So we are taking a example of stateful wordcount in socketTextStreaming. Like in…

Data Science & Spark:- Data visualization for processing out the key factors using Zeppelin.

What is it a data scientist yearn for most? For me it’s Data Visualization. We know that under those massive layers of junk there is always something important hidden that…