Arvind Pant – Medium

Arvind Pant

Arvind Pant

Masking PII (Personally Identifiable Information)

As a data engineer, you’ve likely encountered databases or tables to which you do not have access without obtaining multiple approvals…

May 29

Masking PII (Personally Identifiable Information)

May 29

Arvind Pant

Spark Job Performance Issue: Debugging & Solution

Introduction: In the realm of big data processing, Spark jobs are often hailed for their efficiency and scalability. However…

May 1

Spark Job Performance Issue: Debugging & Solution

May 1

Arvind Pant
in
DevOps.dev

Interview on System Design for Data Engineers

Introduction:

Apr 13

Interview on System Design for Data Engineers

Apr 13

Arvind Pant
in
Towards Dev

Clickhouse Ingestion

Introduction to Clickhouse ClickHouse stands out as a high-performance columnar database tailored for rapid data retrieval and efficient…

Apr 6

Clickhouse Ingestion

Apr 6

Arvind Pant

Exploring Delta Lake Optimization Parameters

Delta Lake offers several optimization parameters that enhance the performance and efficiency of data storage and processing. In this blog…

Mar 30

Exploring Delta Lake Optimization Parameters

Mar 30

Arvind Pant
in
DevOps.dev

Spark Streaming Jobs: Monitoring and Alerting for Silent Failure Part 2

This is the second part of the series. In the first part, we explored how to integrate metrics into our Spark streaming code. In this…

Mar 17

Spark Streaming Jobs: Monitoring and Alerting for Silent Failure Part 2

Mar 17

Arvind Pant

Spark Streaming Jobs: Monitoring and Alerting for Silent Failure Part 1

Introduction: In the fast-paced world of data streaming, ensuring the reliability of your streaming jobs is crucial. However, there are…

Mar 9

Chronosphere Architecture: source chronosphere

Mar 9

Arvind Pant

Best Practices in Programming

Introduction

Mar 4

Mar 4

Arvind Pant

Databricks Snowflake Connector: Snowflake to Delta Transfer Issue

This is very interesting issue we faced during data transfer from Snowflake to Delta using Snowflake Connector

Mar 2

Databricks Snowflake Connector: Snowflake to Delta Transfer Issue

Mar 2

Arvind Pant

Arvind Pant

I'm passionate about continuous learning and sharing my insights with others. Writing allows me to offer guidance and learn from different perspectives.

Following

Help
Status
About
Careers
Press
Blog
Privacy
Terms
Text to speech
Teams