Thomas ThomasHow to Flatten Json Files Dynamically Using Apache Spark(Scala Version)There are several file types are available when we look at the use case of ingesting data from different sources. Some of them are Parquet…Mar 2, 20226Mar 2, 20226
Thomas ThomasHow to UPSERT data into a relational database using Apache Spark: Part 2 (Python Version)In my opinion, Database UPSERT won’t be complete without talking about MERGE functionality. In this blog, we will sail through how we can…Feb 15, 20226Feb 15, 20226
Thomas ThomasHow to UPSERT data into a relational database using Apache Spark: Part 1(Python Version)Apache Spark has multiple ways to read data from different sources like files, databases, etc. But when it comes to loading data into…Feb 13, 20221Feb 13, 20221
Thomas ThomasHow to Flatten Json Files Dynamically Using Apache PySpark(Python)There are several file types are available when we look at the use case of ingesting data from different sources. Some of them are Parquet…Feb 5, 20227Feb 5, 20227
Thomas ThomasIntegrate C/C++ Libraries(dll/so) into Apache Spark/Scala in Hadoop ClusterC++ is a powerful, efficient and fast language. C/C++ is the backbone of all the well-known operating systems, browsers, Machine learning…Jul 26, 20202Jul 26, 20202
Thomas ThomasHow to load millions of data into Mongo DB using Apache Spark 3.0Mongo DB is a distributed NOSQL(Not Only SQL) database based on a document model where data objects are stored as separate document inside…Dec 11, 20191Dec 11, 20191
Thomas ThomasHow to create Spark Dataframe on HBase table.Apache HBase and Hive are both data stores for storing unstructured data. HBase is a distributed, scalable, NoSQL big data store that runs…Nov 26, 20192Nov 26, 20192
Thomas ThomasHow to UPSERT data into relational database using Apache Spark: Part 2In my opinion, Database UPSERT won’t be complete with out talking about MERGE functionality. In this blog we will sail through how we can…May 18, 20194May 18, 20194
Thomas ThomasHow to Upsert data into relational database using Spark.Spark has multiple ways to read data from different sources like files, databases etc. But when it comes to loading data into…May 3, 20198May 3, 20198
Thomas ThomasMost Common Scala Collections used in SparkThe latest Scala release has its new and improved collection library. Scala collection library is very practical and useful.In this blog I…Apr 30, 2019Apr 30, 2019
