Python Basic Data Loading Methods
There are many cases where you would like to load data from many sources for Data Analysis or Data Processing.
Much of what I am covering is captured in the notebook You can download it from the below link:
Importing data from CSV¶
The native python csv module can be used to load a csv file. Sample code is given below.
Importing data from Microsoft Excels
In python there are many ways to load the data from Excel, in the below code I use the xlrd library to load data from excel. Please make sure to install xlrd library.
Importing data from fixed-width data files
Fields are of fixed widths and we can infer the format to match and extract data.We can load such files using struct library which is part of standard python distribution.
Importing data from tab-delimited files
Loading from tab delimited file can again be done using python csv module.
Importing data from a JSON resource
Python’s native json module can be used and also many 3rd party libraries have support to get json data. Here we see the jason output from requests module which is used after we parse output from the Github API.
Exporting data to JSON, CSV, and Excel
Now apart from the libraries used above we will be using xlwt library to write to an Excel file. Here we read a fixed length file and then write it to cvs, json and excel.
Importing and manipulating data with Pandas
Now Pandas is a very versatile library and widely used for data analysis for large datasets which can fit into memory. Here we show how we can load a data to Pandas, add one column and easily save the file.
Importing data from a database
Here we use SQLite module to see how to interact with a database for simple operations.Here I show how you can create a database using an sql file and query the DB to get some data.
Hope this information is helpful, much of the data files are there in github repo. Feel free to contact me at firstname.lastname@example.org or follow me at twitter.com/twitmyreview