Become a member
Sign in
ali mesut karadeniz
ali mesut karadeniz

ali mesut karadeniz

2 Following
1 Followers
  • Profile
  • Claps
  • Highlights

Highlighted by ali mesut karadeniz

See more

From Why and How to Use Pandas with Large Data by Admond Lee

The parameter essentially means the number of rows to be read into a dataframe at any single time in order to fit into the local memory. Since the data consists of more than 70 millions of rows, I specified the chunksize as 1 million rows each time that broke the large data set into many smaller pieces.

From Why and How to Use Pandas with Large Data by Admond Lee

Here comes the good news and the beauty of Pandas: I realized that pandas.read_csv has a parameter called chunksize!

From Why and How to Use Pandas with Large Data by Admond Lee

To be honest, I was baffled when I encountered an error and I couldn’t read the data from CSV file, only to realize that the memory of my local machine was too small for the data with 16GB of RAM.

Claps from ali mesut karadeniz

See more

Kavram karmaşası üzerine

Canburak Tümer

Yazılımda Kırık Camlar

Canburak Tümer

Very simple Python script for extracting most common words from a story

Tirthajyoti Sarkar