Importing Data In Python

In this blog post, I’ll tell you about import operations with python.

Txt Files (.txt)

import numpy as np
filename = 'MNIST_header.txt'
data = np.loadtxt(filename, delimiter=',', skiprows=1, dtype=str)
print(data)

Csv Files (.csv)

import pandas as pd
data = pd.read_csv(file, sep='\t', comment='#', na_values=['Nothing'])

Pickle Files (.pkl)

import pickle
with open('data.pkl', 'rb') as file:
    d = pickle.load(file)
print(d)

Excel Files ( .xlsx )

import pandas as pd
file = 'datafile.xlsx' 
data = pd.ExcelFile(file)
print(data.sheet_names)

Statistical Analysis System (SAS Files (.sas7bdat))

import pandas as pd
from sas7bdat import SAS7BDAT
with SAS7BDAT('urbanpop.sas7bdat') as file:
df_sas = file.to_data_frame()

Academic Social Sciences Research (Stata Files ( .dta ))

import pandas as pd
data = pd.read_stata('urbanpop.dta')

HDF5 Files (Hierarchical Data Format version 5)

import h5py
import h5py
filename = 'H-H1_LOSC_4_V1-815411200-4096.hdf5'
 
data = h5py.File(filename, 'r') # 'r' is to read 
print(type(data))

MATLAB Files (.mat File)

import scipy.io
filename = 'example1.mat'
mat = scipy.io.loadmat(filename)
print(type(mat))

Relational Database Management Systems

PostgreSQL

MySQL

SQLite

We practice the following steps for relational database management systems.

1)Import packages and functions

2)Create the database engine

3)Connect to the engine

4)Query the database

5)Save query results to a DataFrame

6)Close the connection

Pandas with SQL Query

from sqlalchemy import create_engine
import pandas as pd
engine = create_engine('sqlite:///Turkey.sqlite') 
with engine.connect() as con:
rs = con.execute("SELECT * FROM Table") df = pd.DataFrame(rs.fetchall()) 
df.columns = rs.keys()
df = pd.read_sql_query("SELECT * FROM Table", engine)
con.close()

Inner Join

I’ll just tell you the example of inner join. Let’s go!

There are two tables called “Orders” and “Customer”. In these two tables, the common columns are called join. “CustomerID” is common in both tables.

Inner Join in Python with Pandas

from sqlalchemy import create_engine
import pandas as pd
engine = create_engine('sqlite:///Northwind.sqlite')
df = pd.read_sql_query("SELECT OrderID, CompanyName FROM Orders
INNER JOIN Customers on Orders.CustomerID = Customers.CustomerID", engine) print(df.head())

That’s all!