How to easily retrieve minute level price & volume data from Binance for free.
Background
If you have been following my posts, I have been attempting to do quantitative analysis in the markets — Building a Momentum Trading Strategy for Crypto. In order to come up with good quantitative analysis and strategies, we need to be able to test and validate them thoroughly. Therefore, it is of the utmost importance to have clean and reliable data that we can work with.
In this article, we will see how to retrieve the entire minute level historic price and volume data of BNB/USDT (Binance Coin — 3rd largest market cap coin) with the Binance API.
Why Binance?
Binance is selected as it is currently (2021 Nov) the largest crypto exchange in the world by 24h trading volume according to CoinMarketCap.
Another reason is because Binance has a very generous limit on its open API. This gives us a theoretical limit of retrieving 13 days worth of minute level price volume data from Binance for a trading pair. Which means we only need 27 seconds to retrieve each year’s worth of price history. However, we will only be using single core for simplicity, which means the performance will only allow 16 hours worth of data per second (9 minutes / year of data / trading pair).
Calculation — If you must know… (1req can retrieve 1,000 minute level rows)
1200req/min = 20req/sec =20*1000minute-row/req*req/sec=13.89day-rows/sec=1year-rows/27secs
Understanding the Binance API
We will use Binance’s Kline/Candlestick Data API endpoint. It takes 5 parameters and gives 12 return figures including OHLCV explained below.
Input Parameters
- symbol — BNBUSDT
- interval —“1m” representing 1 minute
- startTime —UTC in ms. Will be set dynamically (ex. 1636710431000)
- endTime — Start from midnight yesterday and go back by 13 days each
- limit — 1000 for maximum efficiency
Output Returns (for BNBUSDT symbol by minute)
- Open Time & Close Time —Different by minute -1microsecond
- OHLC— Price information in USDT
- Volume — Volume in BNB
- Number of trades
- Quote asset volume — Volume in USDT
- Taker buy base asset volume — Volume in BNB of Taker Buy(=Maker Sell) — i.e. volume = “taker buy base asset volume” + “taker sell base asset volume” => We can also get Taker Sell / Maker Buy / Maker Sell in BNB
- Taker buy quote asset volume — Volume in USDT of Taker Buy
- Ignore — literally ignore
CODE
Initial Setup
First, set the fixed variables including the cryptocurrency symbols you want to retrieve the history of. For this project the quote is fixed to USDT for simplicity but you can change to any quote you wish in the next section.
Getting the Data
Now it’s the essense of this article. Use the Binance API to get 1000 minute level price data per request. I have included the print statements for logging so you can keep track of how the process is going. Note that intermediary data will be stored for each year so that we do not lose any progress.
Combining the Data
Since our CSV output files will be divided by year, let’s combine them to one single file that we can use. At the same time let’s add symbol (BNB)and currency (USDT) to the data so we do not have to only rely on the filename.
Conclusion
That’s it! The process may take some time depending on what kind of trading pair you are requesting. For tokens with short trading history (e.g. Solana) the process won’t take much time. However, for Bitcoin it may take more than an hour.
If you are proficient in Python, you can use threading to significantly improve the performance of the task as it is network-bound, and the API limit is far above (>20x) what the above code uses. For the sake of this article I have not introduced it to keep things simple.
If this helped, give some claps and hit follow! Don’t miss out on further tips and insights!
