Web Scraping with Python in Indonesian E-Commerce (Tokopedia) Part 2: Present the Data as a CSV File
Comma-separated values or usually called CSV is a file format that uses comma (,) to separate values. So it’s quite different with Excel files that separate the value with cell if we open it with Excel file openers like Microsoft Excel or Google Sheets.
CSV file format is widely used in the tech industry nowadays. And I like it too because it is quite simple to differentiate the column (Except your data value has a comma too) and you can open it the same way as ordinary text file format like txt.
So now we will represent the data that we already get before to CSV file format.
Let’s create a file named csv_creator.py
as a file where the class that generates CSV resides.
Then let’s create a class named CSVCreator
that has a responsibility to create a CSV file from the data provided.
import csv
class CSVCreator:
After that, we will create a static method in that class so we don’t need to create objects from the class every time we want to generate the CSV file. And let’s call the method create_csv
. This method will receive the data generated by the Scraper
object.
@staticmethod
def create_csv(datas):
As we declared before, the data generated by the Scraper
will have dictionary form and take name
, price
, city
, and img
as their key. So let's define the key, it will be useful later.
fields = ["name", "price", "city", "img"]
To write the data into a file at the disk in Python 3, of course, we need to create an IOWrapper
. We can create that by using this code.
with open('shoes.csv', 'w') as f:
Then, let’s use the csv
library to write our data into the shoes.csv
file. First, we need to create an DictionaryWriter
object from the csv
library. This action is needed because the shape of each of our data is a Python 3's dictionary. And we will input our fields
variable to the DictionaryWriter
an object so they can recognize the key of the data that need to be written to the file. We can create the object with this code.
writer = csv.DictWriter(f, fieldnames=fields)
The CSV file is usually shaped in two-part, the header and the value. This is how we can recognize what kind of value is in each column. In this code, we will write the header and all the data to the file too.
writer.writeheader()
writer.writerows(datas)
And that’s all for our CSVCreator
class. For all the code you can check this.
import csv
class CSVCreator:
@staticmethod
def create_csv(datas):
fields = ["name", "price", "city", "img"]
with open('shoes.csv', 'w') as f:
writer = csv.DictWriter(f, fieldnames=fields)
writer.writeheader()
writer.writerows(datas)
In the main.py
file, we can delete the part that prints all the data to the terminal and change it to the CSVCreator
and call the create_csv
method. So the code will be like this.
from scraper import Scraper
from csv_creator import CSVCreator
if __name__ == "__main__":
scraper = Scraper()
datas = scraper.get_data()
index = 1
CSVCreator.create_csv(datas)
And if we run the main.py
file, we can see that shoes.csv
file was created and has data like this.
You can check the overall step on my blog with this link or the next step on medium with this link.
If you want to discuss something just contact me on my LinkedIn.
Thank you very much and goodbye.