Crypto Order Book: Part I, Bitfinex.
Looking at the landscape of cryptocurrency tradings (as well as traditional assets tradings), we all know that analysis of the states of an order book for a given asset is very critical and could potentially lead to discovery of important trading signals. As a crypto-tick-data provider (https://github.com/cryptochassis/cryptochassis-api-docs), we would like to take this opportunity to reveal the arduous journey that we had taken in order to produce a correct and validated order book and highlight the pitfalls and gotchas for each exchange’s REST and websocket APIs related to order books. In this part, let us start with a very popular one: Bitfinex.
Their documentation is https://docs.bitfinex.com/reference#ws-public-books. Very straightforward to use:
wscat -c wss://api-pub.bitfinex.com/ws/2
{ "event": "subscribe", "channel": "book", "symbol": "tBTCUSD" }Parsing the received websocket message is also very straightforward. However, here we encountered the first important issue: there aren’t any timestamp field in the response. This is important from a data collection (and thus subsequent data mining) point of view. Most other major exchanges clearly provide a EVENT timestamp in their websocket API: for trades, it represents at exactly what time the trade happened, and for order book updates, it represents at exactly what time the order book changed. The solution is “hidden” in https://docs.bitfinex.com/docs/ws-general. There is a section called “Configuration”.
To enable timestamp, we simply do this:
wscat -c wss://api-pub.bitfinex.com/ws/2{ "event": "conf", "flags": 32768 }
{ "event": "subscribe", "channel": "book", "symbol": "tBTCUSD" }
Now our response contains a timestamp field:
[198,[10335,2,1.25],1581721279360]In the example websocket message pasted above, 1581721279360 is the unix timestamp in milliseconds.
Then we observed a second problem associated with the timestamp: it is client-dependent. You could easily verify this by opening two terminals on your PC and compare the timestamps side-by-side. The observation was also directly confirmed by Bitfinex: https://gist.github.com/prdn/b8c067c758aab7fa3bf715101086b47c#gistcomment-3136626. If you trading algorithm is highly time-sensitive (e.g. high-frequency trading), you may want to pay extra attention to that timestamp field.
The last (but not least) important discovery on Bitfinex’s websocket API is its sequence number.
Sequence number becomes very critical if your trading algorithm needs to access the current state of the reconstructed order book from the websocket messages. In short, it is a sequence of gapless, monotonically-increasing integers, and it commonly used by the client to verify that they have never missed a single message and they have received them in the correct order. Several other exchanges also provide sequence number in their websocket feeds (e.g. https://docs.pro.coinbase.com/#sequence-numbers, https://docs.gemini.com/websocket-api/#sequence-numbers). Bitfinex also officially provided sample code to illustrate how to check sequence number: https://gist.github.com/prdn/b8c067c758aab7fa3bf715101086b47c#file-bfx_test_book-js-L115. For a long running process such as websocket connection streaming, make sure to check the validity of sequence numbers because otherwise you’ll end up with incorrect reconstructed order books (https://github.com/altangent/ccxws/issues/76#issuecomment-510633429).
With all those said, if you are trying to collect order book tick data for historical data warehousing purposes or algorithmic trading back-testings, we provide a public service (https://github.com/cryptochassis/cryptochassis-api-docs) for major crypto tick data, so that you could focus on your own great projects by standing on our shoulders. :)
