Web Scraping with Python in Indonesian E-Commerce (Tokopedia) Part 2: Present the Data as a CSV File

Comma-separated values or usually called CSV is a file format that uses comma (,) to separate values. So it’s quite different with Excel files that separate the value with cell if we open it with Excel file openers like Microsoft Excel or Google Sheets.

CSV file format is widely used in the tech industry nowadays. And I like it too because it is quite simple to differentiate the column (Except your data value has a comma too) and you can open it the same way as ordinary text file format like txt.

So now we will represent the data that we already get before to CSV file format.

Let’s create a file named csv_creator.py as a file where the class that generates CSV resides.

Then let’s create a class named CSVCreator that has a responsibility to create a CSV file from the data provided.

import csv

class CSVCreator:

After that, we will create a static method in that class so we don’t need to create objects from the class every time we want to generate the CSV file. And let’s call the method create_csv. This method will receive the data generated by the Scraper object.

@staticmethod
def create_csv(datas):

As we declared before, the data generated by the Scraper will have dictionary form and take name, price, city, and img as their key. So let's define the key, it will be useful later.

fields = ["name", "price", "city", "img"]

To write the data into a file at the disk in Python 3, of course, we need to create an IOWrapper. We can create that by using this code.

with open('shoes.csv', 'w') as f:

Then, let’s use the csv library to write our data into the shoes.csv file. First, we need to create an DictionaryWriter object from the csv library. This action is needed because the shape of each of our data is a Python 3's dictionary. And we will input our fields variable to the DictionaryWriter an object so they can recognize the key of the data that need to be written to the file. We can create the object with this code.

writer = csv.DictWriter(f, fieldnames=fields)

The CSV file is usually shaped in two-part, the header and the value. This is how we can recognize what kind of value is in each column. In this code, we will write the header and all the data to the file too.

writer.writeheader()
writer.writerows(datas)

And that’s all for our CSVCreator class. For all the code you can check this.

import csv

class CSVCreator:

  @staticmethod
  def create_csv(datas):
    fields = ["name", "price", "city", "img"]

    with open('shoes.csv', 'w') as f:
      writer = csv.DictWriter(f, fieldnames=fields)

      writer.writeheader()
      writer.writerows(datas)

In the main.py file, we can delete the part that prints all the data to the terminal and change it to the CSVCreator and call the create_csv method. So the code will be like this.

from scraper import Scraper
from csv_creator import CSVCreator

if __name__ == "__main__":
  scraper = Scraper()

  datas = scraper.get_data()
  index = 1

  CSVCreator.create_csv(datas)

And if we run the main.py file, we can see that shoes.csv file was created and has data like this.

You can check the overall step on my blog with this link or the next step on medium with this link.

If you want to discuss something just contact me on my LinkedIn.

Thank you very much and goodbye.