Trading in the Cloud: Market Microstructure Considerations
Does a US Equities agency broker need ultra-low latency?
Recently, we wrote about the technical feasibility of running a latency-sensitive US Equities algorithmic trading system in the cloud. In this article, we talk about the market microstructure implications of this. More specifically, we address the question: “Doesn’t US Equities trading require extremely low latencies to the exchanges?”
Important side note: Even though we are posting this now, this topic may be a moot point in a few months, once we move our trading system to the recently-announced AWS New York Local Zone (which is physically located in New Jersey). This move will put our system at a level footing with any agency broker on-prem system when it comes to geographical latency.
Background
Our system runs in the us-east-1 region of AWS, which is physically located in N. Virginia and is about 3.5 milliseconds of network latency away from the major trading centers in New Jersey.
There is a widely held belief that unless you spend like 10 million dollars getting as close as possible to the exchanges, and subscribe to the fastest/most expensive connectivity lines and feeds, you can’t really “compete”. We think these efforts to get ultra-fast are misguided when it comes to agency brokers trading on behalf of clients. We’ve discussed why our current geographical latency is acceptable to us in the original post describing our platform and we even have a whole dedicated post on this. However, given the continued interest in this topic, we’ll try to go over some of the points again in more detail.
Conversation with a Low Latency Enthusiast
Just because there are so many points to cover, and to make this a bit more readable, I’ve turned this into a dialogue with a hypothetical low-latency trading engineer/enthusiast (LLTE).
Caveat: If it wasn’t already clear, we are talking about this from the perspective of an agency US equities trading system. Different considerations apply for systems that are used for proprietary trading, market-making, or exchanges. And of course, it all depends on the asset class and the region as well.
Me: We’ve created an equities trading system that runs entirely in the cloud!
LLTE: You can’t really run anything latency-sensitive in the cloud. And even if you could do that, you’re so far away from the exchange — what is it, like, 4 milliseconds? That’s an eternity! How can you compete — do you even have the latest price?
Me: You should read some of our blog posts! I agree 4ms seems like a long time, but you’d be surprised. Stock prices actually don’t change that often, and there are vast periods of time in between these changes when they are stable. So, if you’re not reacting to quote changes and attempting to trade during the brief periods when prices are changing, the NBBO (National Best Bid Offer) is typically stable at the millisecond level.
LLTE: Don’t the prices jump around at the microsecond level?
Me: Sometimes, but usually not really. Here are some recent stats — for a sampling of dates in 2021, I calculated the percent of unique milliseconds containing an NBBO price change for SPY and IWM (two busy symbols). For SPY, only ~0.7% of the milliseconds in the trading day contained NBBO price changes (the average gap between NBBO price changes = 148ms; median 10ms). For IWM, this number is ~0.6% (the average gap between NBBO price changes = 169ms; median 22ms). For most stocks, the number of such milliseconds is actually much lower, and the average across all stocks that we traded in 2021, other than SPY/IWM, is only 0.03% (the average gap between NBBO price changes = 3017ms; median 49ms).
LLTE: Well, price changes are often clustered together, so looking at unique milliseconds with price changes may be misleading.
Me: Is it though? If our trading logic avoids these clustered periods by not reacting immediately to price changes, we can avoid the speed race during these milliseconds. Ok, how about some direct Proof stats. Of the roughly 650,000 orders that we sent to the venues in 2021, only about 0.5% of orders experienced an NBBO price change while the order was in flight (that’s 1 in ~200 orders). And note that not all price changes are bad — some of these were beneficial for the client’s order (e.g. price improving while taking liquidity), while some had no meaningful effect (e.g. when sending pegged orders).
LLTE: Interesting. But it’s not just price changes while in flight — when you try to take liquidity across multiple exchanges, the market would detect that and the price would change in the middle of your sweep.
Me: Yeah, probably true. We avoid this issue by using an external router for lit sweeps. The IEX router manages to give us a nearly 100% fill rate when sweeping exchanges. We believe in using the best utilities and innovations available to us across the street. The IEX router is one of them — it’s inexpensive, works well, and we happen to trust it because we know exactly how it works.
(disclosure: members of the proof team built the IEX router when we were back at IEX)
LLTE: Ok, how about posting orders? You could never compete with the faster participants. Your orders would be way down in priority in the order queue.
Me: Well, depends on what your goal is? If your goal is to harvest rebates and you want to get near the top of the queue at the exchanges that pay the most rebates, you do need to be fast. However, if you’re willing to post on non-maker-taker exchanges where you would have a naturally higher priority in the intermarket queue, you could still achieve a better outcome for your customers than blindly joining the queue at the busiest venue. Let me ask you this — what is your posting logic for lit orders?
LLTE: We mostly join the inside bid or ask at a few venues — the primary exchange and a few more. We use historical data and real-time heatmaps to determine where to post. We also layer the orders so we have some orders a tick or more outside the NBBO.
Me: Hmm, so you do join the bid or ask, meaning that there are people ahead of you in the order queue by definition? And you’re probably not the absolute fastest participant, so there are probably multiple other participants ahead of you? Haven’t we all heard things like — “The arms race is a winner take all game. There is no second place price”. But your posting logic is designed to place you in at least the second place or even further behind?
LLTE: That’s… you’re twisting words, that’s not how it works. There is definitely an advantage to being fast.
Me: Ok, look, I’m not saying latency doesn’t matter at all, but there isn’t one perfect way to do this in all circumstances. As an agency broker, your Algo/SOR decision-making component is not likely to be co-located with every exchange. No matter where it is in NY/NJ, you are likely at least 200 microseconds away from any particular exchange (e.g. if you’re in Secaucus, you’re near CBOE, but away from NYSE and Nasdaq). Let’s face it — no matter how fast, an agency broker is unlikely to do well at this game.
LLTE: Well, we use C++ and 40Gbps lines and binary order entry protocols so that we are fast and we can post on the exchanges where there is most liquidity. Really if you don’t post on the largest exchanges, you could never get filled enough.
Me: That depends on how much you’re trying to trade at a time (as a percentage of ADV), and plus, I never said we don’t post on the primary exchange where the volumes are typically the highest. I’m saying you can do this smartly (allocate between venues to balance between fill rates and execution quality) and achieve similar or better outcomes than just being fast. And as long as we’re talking about finding enough liquidity, we can’t ignore dark venues, where other factors besides latency are much more important (e.g. counterparty filtering or order types).
LLTE: No matter where you post, one reason you need low latency is to get out of the way quickly when the quote is changing, otherwise you would suffer from adverse selection.
Me: First, have you checked that you can actually do this reliably? Second, the IEX D-Limit order type allows us to do exactly that without scrambling to send a cancel/replace every time the stock reprices. We routinely see situations where we’re resting on D-Limit + the primary exchange, the market moves into us and we get filled at the primary; a moment later, we get filled on the D-Limit order too but with multiple ticks worth of price improvement. The 1-second markouts bear this out: D-Limit consistently and significantly outperforms NYSE/Nasdaq/ARCA.
(disclosure: members of the proof team played a key role in creating the crumbling-quote indicator that powers D-Limit)
LLTE: Do you not trade anywhere other than IEX?
Me: We do, of course. We trade on all exchanges and multiple dark pools, but as I said before, we like to use the full set of investor-friendly tools available on the street (and not just from IEX; another example is the M-ELO order type at Nasdaq; there are others, especially in the ATS space). We may not be the fastest, but we want to be the smartest and that can go farther than being brute-force fast.
LLTE: Meh, I still think you need to be fast to compete.
Me: Hmm… you keep saying “compete”. What is the business you think we’re competing in? I’ll tell you our answer — we want the best outcomes for large (full-day, sometimes even multi-day) orders for our institutional clients. And that requires doing a good job at the macro strategy level along with the micro tactics level. In algorithmic trading, there are 3 things we need to be good at deciding: when to trade, how much to trade, and how to trade. The first two are macro intelligence and do not require low latency. Macro can trump micro, and make micro a lower order term in the overall trade performance. It is entirely possible for an order to get so-called “good” fills but suck at the overall performance because it traded unevenly during the day, or traded too much at any given time, or traded at inopportune times.
LLTE: That all makes sense. But I am told you need to be fast to compete. We use kernel bypass.
Me: That’s… great. Good talk then!
Closing Thoughts
We believe we have struck the right balance between focusing on the high-level algo intelligence and the low-level tactics, which we expect will lead to superior execution performance for our clients. As mentioned in our recent 3-year update, we have just released our flagship algo (simply named “Proof”), and we no longer consider our trading platform to be “in beta”. We are now ready to onboard institutional clients beyond our initial pool of pilot customers.
If you have counter-arguments or feedback on any of the things we are doing, please reach out to me on Twitter: @preraksanghvi, or drop us a line at info @ prooftrading.com.
If you think we’re doing cool stuff and you would like to contribute, reach out to us at careers @ prooftrading.com. See current open roles here.
