Open in app
Home
Notifications
Lists
Stories

Write
Harmeet Sokhi
Harmeet Sokhi

Home

Nov 20, 2021

A small attempt to reduce landfill waste

Last year in August 2020, I finally decided that I needed to do a better job at reducing waste and do my part. For years I had just been thinking about it and kept feeling guilty but there was an absolute lack of action and a lot of procrastination. …

Compost

4 min read

A small attempt to reduce landfill waste
A small attempt to reduce landfill waste

May 5, 2020

How to create a local airflow pipeline with PySpark, LocalStack S3

This article will help you setup an apache-airflow pipeline locally on your machine which would run a spark job by reading and writing from AWS s3 bucket. Following are the highlights before we move into the details Spark is spun up using airflow dockeroperator Localstack has been used to create…

Spark

3 min read

How to create a local airflow pipeline with PySpark, LocalStack S3
How to create a local airflow pipeline with PySpark, LocalStack S3

Apr 20, 2020

Spark Optimization Checklist

When I started working on spark there were some basic optimisation tricks which I wish I knew .It took me some time to research and figure out. There is a lot of material available on optimisation techniques but It takes a while for a newbie to find quick spark optimisation…

Spark Optimization

3 min read

Spark Optimisation Checklist
Spark Optimisation Checklist
Harmeet Sokhi

Harmeet Sokhi

Data,ML and Tech

Following
  • Simon Aubury

    Simon Aubury

  • Sandeep Uttamchandani

    Sandeep Uttamchandani

  • Brad Nguyen

    Brad Nguyen

  • Jeremiah Hansen

    Jeremiah Hansen

  • Neeti Arya Bhutani

    Neeti Arya Bhutani

Help

Status

Writers

Blog

Careers

Privacy

Terms

About

Knowable