Retrieving Full Historical Data for Every Cryptocurrency on Binance & Bitmex Using the Python APIs

A single function to read, update, save and gather data.

Peter Nistrup
Jul 17, 2019 · 4 min read
Image for post
Image for post

Disclaimer: I know this isn’t what I typically post but since it’s an interest of mine — just like traditional finance, machine learning, and data science — I think some of you might find this interesting as well.

I’ve recently been investigating the relationship between data science and the cryptocurrency market for a pretty long article I’m writing. During the process, I needed to retrieve price history and other data, so I decided to write a supplementary piece about how I accomplished that.

I hope you find it useful. Make sure to if you enjoy this piece and want to see more!

TL;DR

To retrieve Bitcoin prices and data (1m klines):

  1. Sign-up on and/or to get API access.
  2. Import the functions I’ve created for easy-of-use and add your API details.
  3. a. Call the function: get_all_binance(“BTCUSDT”, “1m”, save = True)
    b. Call the function: get_all_bitmex(“XBTUSD”, “1m”, save = True)

Example use:

Image for post
Image for post
An example of getting 5m historical price data of EOS on BitMex using the get_all_bitmex function
Image for post
Image for post
An example of getting 5m historical price data of ICX on Binance using the get_all_binance function

Why Binance and BitMex?

If you’ve had any experience with the cryptocurrency market you know that the answer to this question is obvious:

Binance is the largest exchange on the market and Bitmex is the largest ‘indirect’ volume mover.

The reason I emphasise ‘indirect’ is because BitMex isn’t actually trading crypto, but rather extreme leverage / margin contracts, effectively making it a huge market mover.

As we know, large volume equals big data and big data (usually) equals better modeling and predictions, especially for machine learning models.

Please note that you DON’T have to buy any Bitcoin or any other cryptocurrency whatsoever to perform the steps in this article. The reason we need to sign up is that once signed up you’ll have access to their API’s, which is what we’re gonna be using for getting the actual data.

You can chose either or both, the functions I’ll introduce below are flexible to both exchanges.

Retrieving and Storing the Data:

Our code will do the following:

  1. Check for existing saved data
  2. Find the difference between the latest saved data (if any) and the newest data
  3. Send API calls and append an existing or new DataFrame
  4. Save and/or return full DataFrame

The only thing you need to do is add your API-key and API-secret to your code, also remember to NEVER share these with anyone. If you are trading or holding cryptocurrency on either or , this could result in a loss of funds.

This is incredibly easy to use, simply call the functions in Python with:

Image for post
Image for post
Example use-case for BitMex

Once downloaded, the function will return the entire DataFrame for you to use as you wish. If the argument save is passed as True, it’ll save the DataFrame on your computer with the filename “[symbol]-[kline_size]-data.csv”, i.e. “XBTUSD-5m-data.csv” in the case above.

The clever thing about this function is that it will keep your data up to date at all time.

Each time the function is called it will check the latest point of data you’ve downloaded and update with any new data!

Just call the get_all_bitmex(“XBTUSD”, “5m”, save = True) each time you’re working with your XBTUSD data to make sure it’s up to date!

But I Want Data for All the Markets!

Don’t worry, this is just as easy to do! You can retrieve the data for all markets with:

Here we’re downloading all available data for Bitcoin and its relation to Ethereum and Ripple from both BitMex and Binance.

Conclusion

Thanks for reading. Hopefully, you found this useful!

If you want to see and learn more, be sure to 🔍 and 🐦

The Startup

Medium's largest active publication, followed by +773K people. Follow to join our community.

Peter Nistrup

Written by

DATA SCIENCE, STATISTICS & AI … Twitter: @PeterNistrup, LinkedIn: www.linkedin.com/in/peter-nistrup/

The Startup

Medium's largest active publication, followed by +773K people. Follow to join our community.

Peter Nistrup

Written by

DATA SCIENCE, STATISTICS & AI … Twitter: @PeterNistrup, LinkedIn: www.linkedin.com/in/peter-nistrup/

The Startup

Medium's largest active publication, followed by +773K people. Follow to join our community.

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface.

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox.

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store