How to read utf-8 charachters using Pandas in Python-Machine Learning Course by Hackveda

sameer saxena
Sep 9, 2018 · 2 min read

Pandas is a software library written for the Python programming language for data manipulation and analysis. In particular, it offers data structures and operations for manipulating numerical tables and time series.


Library Feature

  • rame object for data manipulation with integrated indexing.
  • Tools for reading and writing data between in-memory data structures and different file formats.
  • Data alignment and integrated handling of missing data.
  • Reshaping and pivoting of data sets.
  • Label-based slicing, fancy indexing, and subsetting of large data sets.
  • Data structure column insertion and deletion.
  • Group by engine allowing split-apply-combine operations on data sets.
  • Data set merging and joining.

Pre-Requisites

Basic knowledge of dictionaries in python programming.
Basic programming syntax of python


So to start with Pandas library we first need to Import it.

importing pandas library

Reading a CSV file using Pandas

Importing data is the first step in any data science project. Reading a csv ( comma separated values) is hence important.

reading a csv

Reading a text file having encoding of UTF-8

Make sure the file you are importing using pandas is saved in utf-8 format.
In Excel 2016 When saving as: click More Options > Tools > Web Options > Encoding > Save this document as … (pick UTF-8 from the list)

After this we need to do the following

Parameters:
1. Sometimes there are multiple sheets present in the same excel. we can also read it directly by giving the name of the Sheet in the parameter itself. I have created a Sheet2 in the same excel file with some dummy data.
2. Encoding - str, default None
Encoding to use for UTF when reading/writing (ex. ‘utf-8’). List of Python standard encodings

Similarly if you are reading a CSV file same parameter can be used to read the UTF-8 encoded file.


sameer saxena

Written by

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade