Do a Simple Web Scrapping with Python — Part 4: First step

Make our program work functionally

Before we start the coding, we need to understand the concept of our program, so we can look at the global view and the architecture of our program. That’s why I make a flowchart of our program, it also helps us to understand how the program works with all its functions. First, we’ll make the program work with static data, then we’ll add the package to make our program scrape through the source website. Last, we’ll arrange the program structure with a modular architecture so our program looks good and professional!

Model session:

First flowchart model for our program (Function-Based)

As we can see in the model above, our app will work like this:

1. Take the data from the source website.
2. Extract the data into the program.
3. Visualize the data to the client.
4. The data is shown up to the client.

First, we’ll make the basic function of our program, the extract function and the visualize function. In this step, we’ll use the static data(Not real-time data from the source). Because we need to ensure that the program works properly before scraping the source website with the package.

Coding Session:

Open our last project in PyCharm. In this example, my program name is earthquake-info. After your program is opened, right-click on your program folder → New → Python File and name it main. Now we had a new module called main.

We’ll begin to write our code right away!

Write the script above on your main module. The “ _ _if_ _ == ‘_ _ main _ _’: ” script work to allow or prevent parts of code from being run when the modules are imported. Then we write a print() function to print “Main Apps” to the console when we run it.

Right-click the text line → Run ’main’ to run our code. You can see the ‘Main Apps’ is printed successfully.

Next, We’ll make the main function of our program to extract and visualize the data.

Declare an extract_data and the visualize_data function on top of the if-main. Below that, insert the extract_data() to a variable named final_result, we store it in a variable so we can set it in another function parameter. Then call the visualize_data() function and set the parameter as final_result.

Now we’ll insert the extract_data function with static data that we get from the BMKG website. We just need to copy it from the website for temporary

Open the BMKG website, then copy the data of the latest earthquake info to the extract_data function in dictionary form like the script below.

Declare a variable named update with dictionary datatype. Then insert the data to the update variable with key and value form. Return the completed data with the return update statement at the bottom of the function. Now, this function has static dictionary data that we input by ourselves, we’ll upgrade it to live data soon!

Now, we’ll start to work on the visualize_data function. Code it like the script below. After that, you can check it by running the script to see the output

Insert a new variable called result to the function parameter, it will be used to use a key and get the value of a variable with dictionary data that pass in. Then, print the content one by one with string interpolation to get the value from the dictionary.

Now we’ve done! Our main.py module will look like this :

You can check your result by running the code!

Congrats! Now, it can show you the data that you’ve input before! We’ve succeeded to make our program work functionally with the static data. Next, we’ll be using the package so the data will be replaced with the live one!