Python code to get realtime stock prices from Yahoo Finance

Hello Everyone! This article covers information on how to obtain realtime stock prices for a list stock tickers (symbol) and store them in a MySQL table. Before we go into the python code, we need to create a database (if required) and a table to store the downloaded price information from Yahoo Finance webpage.

Yahoo Finance — https://in.finance.yahoo.com/

Step 1 — Create a MySQL table for storing the stock prices

Below set of commands will create a new database called ‘stockdb’ and a table — ‘tickers’.

#Database creation
create database stockdb;
use stockdb;
#Table creation
CREATE TABLE tickers
(id INT NOT NULL AUTO_INCREMENT,
ticker VARCHAR(50) NOT NULL,
price decimal(50,2),
company_name VARCHAR(500) NOT NULL,
listed_exchange varchar(10) NULL,
category varchar(100) null,
updated_on TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (id));

Step 2 — Python code to fetch stock prices from Yahoo Finance

The python program uses the library, ‘BeautifulSoup’ for scrapping the data from the webpage. Please download the necessary libraries using pip command before importing them.

# Import libraries
import requests
from bs4 import BeautifulSoup
import requests_html
import lxml.html as lh
import pandas as pd
import re
from datetime import datetime
from datetime import timedelta
import mysql.connector as sql
import DBcm
import time
import unidecode #used to convert accented words
config = {
    "host": "127.0.0.1",
    "user": "root",
    "password": "root",
    "database": "stockdb",
}

After importing the libraries, read the CSV file which has the list of tickers along with their company names. The file can be downloaded here. Please note that it only contains tickers for Indian stock market (NSE & BSE). This spreadsheet contains data for other markets.

df = pd.read_csv('tickers.csv')
df.head()
df = df.fillna('NA')
ticker_list = df.values.tolist() # change dataframe to list

The below code uses a loop to pass individual tickers. A dynamic Yahoo Finance URL is created with the ticker. Exceptions are handled in few places of the code to avoid errors if in case the stock is not present in Yahoo Finance webpage and also to handle issues while converting the ‘price’ field to a floating-point number. The command, ‘time.sleep(1)’ makes the program to allow 1-second interval before the next loop. Finally, the data is inserted into ‘tickers’ table.

### Extract from Yahoo Link ###
for ticker in ticker_list:
    url = 'https://in.finance.yahoo.com/quote/' + ticker[0]
    session = requests_html.HTMLSession()
    r = session.get(url)
    content = BeautifulSoup(r.content, 'lxml')
    try:
        price = str(content).split('data-reactid="34"')[4].split('</span>')[0].replace('>','')
    except IndexError as e:
        price = 0.00
    price = price or "0"
    try:
        price = float(price.replace(',',''))
    except ValueError as e:
        price = 0.00
    time.sleep(1)
    with DBcm.UseDatabase(config) as cursor:
        _SQL = """insert into tickers
                  (ticker, price, company_name, listed_exchange, category)
                  values
                  (%s, %s, %s, %s, %s)"""
        print(ticker[0], price, ticker[1], ticker[2], ticker[3])
        cursor.execute(_SQL, (unidecode.unidecode(ticker[0]), price, unidecode.unidecode(ticker[1]), unidecode.unidecode(ticker[2]), unidecode.unidecode(ticker[3])))
print('completed...')

The source code can be downloaded from the python notebook file available on GitHub.

This code can also be modified to obtain price/minute for a single stock ticker. Please let me know in the comments if you face any difficulties in implementing the same.

Disclaimer — This code is only meant for learning purpose. You may check Yahoo Finance terms & conditions before downloading the data.

Happy coding! 😃