Suffyan Asad – Medium

Suffyan Asad

Pinned

Suffyan Asad

Spark Essentials: A Guide to Setting up and Running Spark projects with Scala and sbt

A step-by-step look into the process of setting-up, building, packaging and running Spark projects using Scala and Scala Build Tool (sbt)

18 min readJan 27, 2024

--

1

Spark Essentials: A Guide to Setting up and Running Spark projects with Scala and sbt

--

1

Pinned

Suffyan Asad

Spark Essentials: A Guide to Setting Up, Packaging, and Running PySpark Projects

This article covers the process of setting up and packaging PySpark jobs with code files and dependencies, and running them on Spark…

21 min readDec 30, 2023

--

Spark Essentials: A Guide to Setting Up, Packaging, and Running PySpark Projects

--

Pinned

Suffyan Asad

Getting Started with dbt (Data Build Tool): A Beginner’s Guide to Building Data Transformations

Looking to get started with dbt? Check out this beginner’s guide to building data transformations with the Data Build Tool.

19 min readJul 6, 2023

--

Getting Started with dbt (Data Build Tool): A Beginner’s Guide to Building Data Transformations

--

Pinned

Suffyan Asad

Beginner’s Guide to Spark UI: How to Monitor and Analyze Spark Jobs

This article provides a comprehensive beginner’s guide to Spark UI, covering its features and how it can be used to monitor and analyze…

14 min readJun 4, 2023

--

2

Beginner’s Guide to Spark UI: How to Monitor and Analyze Spark Jobs

--

2

Pinned

Suffyan Asad

Handling Data Skew in Apache Spark: Techniques, Tips and Tricks to Improve Performance

Discover how to detect and mitigate data-skew in Spark. Learn about the impact of data-skew and how to detect and fix it!

12 min readJan 30, 2023

--

2

Handling Data Skew in Apache Spark: Techniques, Tips and Tricks to Improve Performance

--

2

Suffyan Asad

Table Partitioning in PostgreSQL: A Beginner’s guide

Dive into PostgreSQL partitioning: A beginner-friendly guide to enhanced query speeds and efficient data management.

18 min readApr 28, 2024

--

Table Partitioning in PostgreSQL: A Beginner’s guide

--

Suffyan Asad

Spark — Leveraging Window functions for time-series analysis in PySpark

Explore time-series analysis in Spark using window functions. Learn to define and apply window functions for insights with code examples.

14 min readNov 27, 2023

--

Spark — Leveraging Window functions for time-series analysis in PySpark

--

Suffyan Asad

Introduction to the English SDK for Apache Spark: Combining the Power of Apache Spark and LLMs

Explore the English SDK for Apache Spark, a powerful tool for data analysis that combines the power of Spark and LLMs.

10 min readOct 15, 2023

--

Introduction to the English SDK for Apache Spark: Combining the Power of Apache Spark and LLMs

--

Suffyan Asad

An introduction to working with JSON data in PostgreSQL

Learn how to work with JSON data in PostgreSQL. This introductory article covers working with JSON data in PostgreSQL with many examples!

12 min readSep 4, 2023

--

An introduction to working with JSON data in PostgreSQL

--

Suffyan Asad

An Introduction to Pandas UDFs in PySpark

Learn how create Pandas UDFs and apply Pandas’ data manipulation capabilities Spark jobs! Introductory article with code examples.

13 min readAug 19, 2023

--

An Introduction to Pandas UDFs in PySpark

--

Suffyan Asad

Suffyan Asad

Data Engineer | Passionate about data processing at scale | Fulbright and George Washington University alum | https://pk.linkedin.com/in/suffyan-asad-421711126

Following

Help
Status
About
Careers
Press
Blog
Privacy
Terms
Text to speech
Teams