How to Extract Bonds Information on Markets Insider via Python? Part 1

The term “corporate bond” is not strictly defined. Sometimes, the term is used to include all bonds except those issued by governments in their own currencies.

Corporate bonds trade in decentralized, dealer-based, over-the-counter markets. In over-the-counter trading dealers act as intermediaries between buyers and sellers.

In this series of articles, we will study how to use Python to extract the bond information in Markets Insider’s bonds finder.

First, let’s study on the structure of its websites before extracting the data.

https://markets.businessinsider.com/bonds/finder
?p=1 (page)
&borrower= (specific company)
&maturity= (shortterm: 0-3 yers / midterm: 3-10 years/ longterm: > 10 years)
&yield= (0: < 5% / 5:5%-10% / 10: 10%-20%)
&bondtype=(6%2c7%2c8%2c19: Corporate/ 2%2C3%2C4%2C16: Government, etc)
&coupon= (0: < 5% / 5:5%-10% / 10: >10%)
&currency= (333: USD / 534: JPY/ 846: SGD, etc)
&rating= (Moody's rating)
&country= (18: USA/ 33: China/ 27/ Singapore, etc)

Step 0: Import all related packages.

from datetime import date
from datetime import timedelta
import pandas as pd
import requests
import sqlite3
from google.colab import drive

drive.mount('/content/drive')
today = date.today()
con = sqlite3.connect('/content/drive/MyDrive/data/Stock.db')

Step 1: Define for-loop to loop through the pages. We are taking 0–3 years, yield between 5% to 10%

for i in range(1, 100):
  url = 'https://markets.businessinsider.com/bonds/finder?p='+ str(i) +'&borrower=&maturity=shortterm&yield=5&bondtype=6%2c7%2c8%2c19&coupon=0&currency=333&rating=&country=18'

Step 2: Send the request to the url and parse the data into dataframe.

res = requests.get(url)
res.encoding = 'big5'
html_df = pd.read_html(res.text)
df =  html_df [0]
df['As_Of'] = today 

Step 3: Insert (or create) the data into SQLite.

df.to_sql('Coporate_Bond_Markets_Insider', con, if_exists='append')

You will find out there is no details like ISIN/ Coupon Payment Date, etc, which are actually located into the hyper link of each item. We will discuss it in the next article :) more to come!

Thank you. If you want to support Informula, you can buy us a coffee here :)

𝗕𝘂𝘆 𝗺𝗲 𝗮 𝗰𝗼𝗳𝗳𝗲𝗲

In Plain English

Thank you for being a part of our community! Before you go:

• Be sure to clap and follow the writer! 👏
• You can find even more content at PlainEnglish.io 🚀
• Sign up for our free weekly newsletter. 🗞️
• Follow us on Twitter(X), LinkedIn, YouTube, and Discord.