Load stock data from TradingView

4 min readDec 23, 2023

Introduction

I gave a talk at Pycon Thailand 2023 about creating programmatic trading. Loading data from price data is one of the topics that the audience found interesting. In the presentation, I mentioned that I scraped the website and imported it into a Python script, but it was only an idea. In this blog, I will take you deep down into the nitty-gritty, and a module I created to let you load the data easily.

TL;DR

You may use the following code to install and run:

Install

# Installation Python version should be >= 3.10
pip install git+https://github.com/batprem/price-loaders.git

Run

from price_loaders.tradingview import load_asset_price


df = load_asset_price(
  "SET:TTB", # asset symbol referenced in TradingView
  3650, # Candle look back
  "1D", # Timeframe
  None # Timezone default to Thai timezone (+07:00)
)
# Output

Note that:

The function load_asset_price consumes 4 arguments:

1: symbol is the asset symbol.

2: look_back_bars is how many bars before the present you want to project. For instance, if you input 3650 with a time frame 1D , this means data from 3650 days before until today will be loaded.

3: timeframe Time frame is the range of time you want a bar to represent. The argument 1D means a day.

4: timezone Timezone

You could stop reading and start trying my module if you want, but if you’re interested in how it works please continue :)

Why don’t we use `requests` or `beautifulsoup` like the other kinds of scraper

The main reason is data fetching to the chart is not running on HTTP(s). Therefore, we can’t use traditional techniques in this case.

If the data is not running on HTTP(s), then how

The answer is it’s running through WebSocket, the protocol which supports data streaming and is widely used in message queues. This is the key that allows TradingView to run more smoothly than others.

Inspection of WebSocket behind the TradingView. Pressing F12 -> Network -> WS

And that we’re going to make a WebSocket connection using Python

I created a repository of a Python module price_loaders available on GitHub https://github.com/batprem/price-loaders. The following snippets are the ways you can interact with it:

Usage:

On your shell, you may run:

“Feel free to use. I created with MIT License”

pip install git+https://github.com/batprem/price-loaders.git

You can run the following Python code to import it.

from price_loaders.tradingview import load_asset_price

# Download pricing data
df = load_asset_price("BTCUSDT", 3650, "1D", None)
df # The dataframe of BTCUSDT

Contribution:

I welcome anyone who wants to contribute to this project. The following snippets guide you.

# Cloning
git clone https://github.com/batprem/price-loaders.git
# I use poetry for package management
poetry install --with dev
# Pre-commit with you lint the code
pre-commit install

What my module does:

Create a WebSocket channel and request the data. In send_message , an input message is hashed for submission.

def request_data(
    symbol: str,
    time_frame: str,
    look_back_bars: int,
    websocket_session,
    chart_session,
):

    time_frame = "1D"
    # look_back_bars = 365 * 10

    resolve_symbol = json.dumps({"symbol": symbol, "adjustment": "splits"})

    # Set up session
    chart_session_name = "price"
    ws = create_connection(
        "wss://data.tradingview.com/socket.io/websocket", headers=headers
    )

    send_message(ws, "set_auth_token", ["unauthorized_user_token"])
    send_message(ws, "chart_create_session", [chart_session, ""])
    send_message(ws, "quote_create_session", [websocket_session])
    ...

2. Listen to the channel. Messages sent back to the client contain price data. We capture it and parse it into a Pandas data frame.

for res in listen(symbol, look_back_bars, time_frame):
    m = res["m"]
    if m == "study_error":
        break
    p = res["p"]
    data = [element for element in p if isinstance(element, dict)][0]
    if not (("price" in chart) and ("price" in data)):
        chart = {**chart, **data}
    if "pe_ratio" in chart:
        break

3. Finally, we get ready-to-use data

You could try it with any other asset like Forex or Cryptocurrency. All of these are gentle introductions to my mini-project. I’d be glad if you guys find it helpful.