ETL stands for Extract Transform and Load. There are a number of ETL tools on the market, you see for yourself here. ETL tools are mostly used for transferring data from one database to another or data warehouse to another, manipulating it such that it’s consistent and etc.. In other words, ETL is the general procedure of copying data from one or more sources into a destination system which represents the data differently from the source(s) or in a different context than the source(s).
8 Skills You Need to Become a Data Scientist | Data Driven Investor
Numbers do not scare you? There is nothing more satisfying than a beautiful excel sheet? You speak several languages…
You could start by cloning/downloading from my github repo. We’ll import our different libraries, our main focus is on petl, pandas and plotly. You can also delete users.csv file, we’ll see how it comes about later… that’s only if you downloaded or cloned from my github repo.
We’ll load our data from the API endpoint
Above is what our users_table variable holds. Notice the nested dictionary objects in address and company. Let’s take interest in address, our aim is to see where our users come from but in a more organized way.
So we Transform. Petl provides a number of methods to transform tabular data, however, we’ll use unpackdict() — just like the name suggests, this method unpacks the dictionary object and sets the key as a column/field name in a table, cut() — this method let’s you choose/specify which columns/fields you are interested in from your table, rename() — this method let’s you rename column/field names
We now have a decent looking table.
So if you check the directory from which you are running your jupyter-notebook you’ll notice a users.csv file.
We can also do exploratory analysis on our csv file. Our aim is to find out where our users are located
Let’s plot on a map using the longitude and latitude points for each user
Since this data was from a random API, those location points are expected otherwise looks like most of our users are mermaids :)
Thank you for following through, I do welcome your feedback
email@example.com, LinkedIn, Twitter- @ElijahAyeeta