Introduction: Hi Everyone,In this Article we will discuss about SCD types.Slowly Changing Dimensions in Data Warehouse is an important concept that is used to enable the historic aspect of data in an analytical system. As you know, the data warehouse is used to analyse historical data, it is essential to…

Important Concepts In Scala. Everyone Who is a beginner in scala face many problems and has many confusions while trying to understand Important concepts in scala. …

Hi Everyone,In this blog I’ve brought together a list of useful Linux commands , which will help you to learn basic unix command. So, I’ve categorised these commands into the following segments. 1. Time and Date commands 2. Unix users commands 3. Text file operations commands. 4. Unix directory management…

Introduction: I’ve never really understood the whole point of checkpointing Spark applications until I’ve recently faced the issue in one of our spark streaming application,which runs continuously and process the data in every 5 min around on a terabyte of data.I have gone through multiple blog posts and StackOverflow questions…

Spark Structure Streaming Introduction to Spark Structure Streaming Spark Streaming is a process in spark where we can process real time data.Here Streaming means continuous ,as soon as we receive the data Spark Streaming will process this data.Streaming divides continuously flowing input data into discrete units for further processing. Spark Streaming is based on Rdd…

This is a Continuous Blog after Part4 on Hive Series(from basics to Advance)(https://medium.com/@patidar.rahul8392/part-4-apache-hive-basics-how-process-structure-data-in-hadoop-8e69375a29ad), If You have not visited on this Blog I suggest Kindly Read my Previous blog and continue with this blog to learn Hive for Big-data. Whenever we design a Big-data solution and execute hive queries on clusters…

This is Continuous Blog after Part3 on Hive (https://medium.com/@patidar.rahul8392/part-3-apache-hive-basics-how-to-process-structure-data-in-hadoop-1af93a3a51c7), If You have not visited on this Blog I suggest Kindly Read my Previous blog and continue with this blog to learn Hive for Big-data. Topics to be covered In this blog. Hive Managed Tables. Hive External Tables. Subqueries In Hive. …

This is Continuous Blog after Part2 on Hive (https://medium.com/@patidar.rahul8392/part1-apache-hive-basics-how-process-structure-data-in-hadoop-5902116da997), If You have not visited on this Blog I suggest Kindly Read my Previous blog and continue with this blog to learn Hive for Big-data. Topics to be covered In this blog. Create Database Create Table Describe Table Show Table Load…

Hello Everyone, This is Continuous Blog after Part1 on Hive (https://medium.com/@patidar.rahul8392/part1-apache-hive-basics-how-process-structure-data-in-hadoop-5902116da997), If You have not visited on this Blog I suggest Kindly Read my Previous blog and continue with this blog to learn Hive for Big-data. System-define Function In hive: There are mainly 3 category of Function in Hive. UDF(User…