Priyam Jain – Medium

Priyam Jain

Priyam Jain

DateTime functions used in PySpark Projects

Recently I got a DM regarding the date functions in PySpark. Hence creating a post on it. Link to Youtube Video : https://lnkd.in/dHHEab6e…

Dec 24, 2023

DateTime functions used in PySpark Projects

Dec 24, 2023

Priyam Jain

Interview Series on PySpark

Suppose you have a table called log_data with the following columns: log_id, user_id, action, and timestamp. Write a SQL query to calculate…

Sep 19, 2023

Sep 19, 2023

Priyam Jain

real-world example of the latest development in data analytics — Delta Live Tables in Databricks!

🌐 Real-World Example: Let’s say you’re tracking user interactions on your e-commerce website. With Delta Live Tables, you can:

Sep 19, 2023

real-world example of the latest development in data analytics — Delta Live Tables in Databricks!

Sep 19, 2023

Priyam Jain

🚀 Unlocking the Power of SQL Window Functions 🚀

In the world of SQL, window functions are the unsung heroes that enable us to perform complex data analysis with ease.

Sep 18, 2023

🚀 Unlocking the Power of SQL Window Functions 🚀

Sep 18, 2023

Priyam Jain

High-Level SQL String Functions with real-world examples! 💼

1️⃣ CONCATENATION: Combine strings with ease using `CONCAT` or `||`. For instance: SELECT CONCAT(first_name, ‘ ‘, last_name) AS…

Sep 17, 2023

High-Level SQL String Functions with real-world examples! 💼

Sep 17, 2023

Priyam Jain

Introducing the Power of ETL with PySpark!

PySpark, the Python API for Apache Spark, has revolutionized the way we handle Big Data. It’s an ETL powerhouse that combines the…

Sep 11, 2023

Introducing the Power of ETL with PySpark!

Sep 11, 2023

Priyam Jain

Magic of “PARTITION BY” and “BUCKET BY” in our data processing journeys. 🚀💻

1. Partition By: a. Use “PARTITION BY” when you have large datasets and want to improve query performance and data filtering based on…

Aug 6, 2023

Magic of “PARTITION BY” and “BUCKET BY” in our data processing journeys. 🚀💻

Aug 6, 2023

Priyam Jain

Apache Spark Transformations Narrow and Wide 🎯

Narrow transformations, also known as narrow dependencies, are a fundamental concept in Apache Spark that play a crucial role in optimizing…

Aug 3, 2023

Aug 3, 2023

Priyam Jain

“Cache,” “Persist,” and “Unpersist” in Apache Spark with practical examples.

1️⃣ Cache: 🔹Caching is the process of storing intermediate data in memory. 🔹Use cache() to keep the data in memory for quick access in…

Aug 2, 2023

Aug 2, 2023

Priyam Jain

Power of PySpark DataFrames: Ways to create dataframe in PySpark 🚀💼

In this post, I’ll show you some super simple ways to create DataFrames in PySpark with real-life examples. Let’s get inspired and learn…

Jul 30, 2023

Power of PySpark DataFrames: Ways to create dataframe in PySpark 🚀💼

Jul 30, 2023

Priyam Jain

Priyam Jain

Motivated and dedicated Data enthusiast with a year of experience identifying efficiencies and problem areas within data streams. Youtube: @pysparkpulse

Following

Help
Status
About
Careers
Press
Blog
Privacy
Terms
Text to speech
Teams