Retrieving Full Historical Data for Every Cryptocurrency on Binance & BitMex Using the Python API
A single function to read, update, save, and gather data
Disclaimer: I know this isn’t what I typically post but since it’s an interest of mine — just like traditional finance, machine learning, and data science — I think some of you might find this interesting as well.
I’ve recently been investigating the relationship between data science and the cryptocurrency market for a pretty long article I’m writing. During the process, I needed to retrieve price history and other data, so I decided to write a supplementary piece about how I accomplished that.
I hope you find it useful. Make sure to follow me if you enjoy this piece and want to see more!
To retrieve Bitcoin prices and data (1m klines):
- Sign-up on Binance and/or BitMex to get API access.
- Import the functions I’ve created for easy-of-use and add your API details.
- a. Call the function: get_all_binance(“BTCUSDT”, “1m”, save = True)
b. Call the function: get_all_bitmex(“XBTUSD”, “1m”, save = True)
Why Binance and BitMex?
If you’ve had any experience with the cryptocurrency market you know that the answer to this question is obvious:
And as we know, large volume equals big data and big data (usually) equals better modeling and predictions, especially for machine learning models.
Please note that you don’t have to buy any Bitcoin or any other cryptocurrency whatsoever to perform the steps in this article. The reason we need to sign up is that once signed up you’ll have access to their APIs, which is what we’re gonna be using for getting the actual data.
You can choose either or both, the functions introduced below are flexible to both exchanges.
Retrieving and Storing the Data
Our code will do the following:
- Check for existing saved data
- Find the difference between the latest saved data (if any) and the newest data
- Send API calls and append an existing or new DataFrame
- Save and/or return full DataFrame
The only thing you need to do is add your API-key and API-secret to your code, also remember to NEVER share these with anyone. If you are trading or holding cryptocurrency on either Binance or BitMex, this could result in a loss of funds.
This is incredibly easy to use, simply call the function in Python with:
Once downloaded, the function will return the entire DataFrame for you to use as you wish. If the argument
save is passed as
True, it’ll save the DataFrame on your computer with the filename “[symbol]-[kline_size]-data.csv”, i.e. “XBTUSD-5m-data.csv” in the case above.
The clever thing about this function is that it will keep your data up to date at all time.
Each time the function is called it will check the latest point of data you’ve downloaded and update with any new data!
Just call the get_all_bitmex(“XBTUSD”, “5m”, save = True) each time you’re working with your XBTUSD data from BitMex to make sure it’s up to date!
But I Want Data for All the Markets
Don’t worry, this is just as easy to do! You can retrieve the data for all markets with:
Thanks for reading. Hopefully, you found this useful!