Trade Smart, Stay Secure: Mastering Encryption and Observability in Financial Markets
It might surprise some to learn that large swaths of financial market data in transit are still unencrypted. Despite advancements in cybersecurity and widespread encryption in other industries, financial trading infrastructure typically prioritizes speed over security. Every nanosecond counts in financial trading, but at what cost? The decision to prioritize speed over encryption is starting to be challenged by changing regulations, but this can cause unintended consequences for performance visibility and market transparency. In this article, we’ll delve into these challenges and what they mean for the future of financial markets.
To set the scene, let’s explore how encryption in financial markets evolved, why it has been a hot topic for performance, and the challenges it has introduced for observability.
A Timeline of Encryption in Financial Markets
- 1990s: HTTPS encryption becomes widely adopted for web browsing, securing internet-based communications.
- 1992: The Financial Information Exchange (FIX) protocol launches to standardize electronic trading. Early versions prioritize functionality over security.
- 2000s: High-frequency trading (HFT) gains momentum, focusing on low-latency connections at the expense of encryption.
- 2009: FIX protocol version 5.0 SP2 introduces encryption standards via TLS, but adoption remains uneven across trading systems.
Prior to FIX 5.0 SP2, encryption support was not standardized, and implementations relied on network-level security measures (e.g., VPNs or proprietary solutions) rather than native encryption within the protocol. Despite the introduction of TLS support, adoption has been slow in many parts of the trading industry, primarily due to concerns about latency overhead and a perceived lack of necessity for encryption in low-latency environments like cross-connects.
The Current State: A Typical FIX Logon Example
Even today, many FIX logon messages transmit credentials in plaintext:
8=FIX.5.2|9=146|35=A|49=TRADER_A|56=BROKER_B|34=1|52=20241113–12:00:00|98=0|108=30|553=USERNAME|554=PASSWORD|10=004|
These credentials are accessible to anyone who views this message as it passes between TRADER_A and BROKER_B. This example underscores a significant vulnerability — one that becomes more critical as cyber threats escalate. In typical implementations, the password is not even hashed and is simply viewable as plaintext on the wire. Usually, this traffic is going over a dedicated cross-connect to the trading venue. However, it still makes the compliance people at the firms and at the exchanges uncomfortable.
Why Hasn’t Encryption Taken Off for Trading?
Encryption in Specific Corners
Encryption initially found its place in customer-facing internet connectivity — like FX single and multi-bank portals — where not only orders but also rate distributions were encrypted. However, when high-frequency trading (HFT) over direct cross-connects emerged, priorities shifted.
Low Latency vs. Security
The HFT era brought an obsession with speed. Cross-connects — physical fiber links between trading counterparties — were deemed secure, making encryption seem unnecessary. Furthermore, encryption’s perceived latency overhead raised concerns. While TLS encryption overhead is minimal (typically a few microseconds), in ultra-low-latency environments, even that was considered too much.
Observability Challenges with Encryption
Encryption complicates observability because once data is encrypted, monitoring tools must decrypt it to make sense of it. For example, encrypted FIX messages obscure fields that are critical for performance and compliance monitoring, such as order IDs or timestamps.
What is the latency overhead of encrypting financial markets traffic? Is it still a genuine issue given newer hardware and technologies?
First of all, let’s define our terms here. Often when we characterise web browser performance, we think about encryption overhead as being the extra delay that is introduced when you first establish a new connection to a server. Encryption adds extra steps here, because the encryption protocol must first of all be negotiated. For financial markets trading, most connections are persistent so we are less concerned with the overhead on connection establishment, and more concerned about the overhead for each packet that needs to be encrypted (to be sent) or decrypted (to be received).
This encryption overhead is generally recognised to be between 1 and 10 microseconds per packet.
That will be acceptable for some financial market participants, but definitely not for all.
However, technologists at firms often have to implement infrastructure that can support multiple different use cases of trading style — so the lowest affordable latency usually wins. This usually means avoiding encryption.
What has changed to accelerate the adoption of encryption?
The narrative is shifting. Firm-wide cybersecurity policies increasingly mandate encryption, leaving trading desks with fewer options to seek exceptions. Simultaneously, governments and regulators are classifying financial infrastructure as critical, demanding higher security standards.
One specific widely-reported incident which may have focused minds was the Bangladesh Bank heist in 2016 also prompted some re-examination of security practice for ‘isolated’ systems that used dedicated infrastructure. Further impetus has been given by the operational resilience initiatives that regulators have pushed since 2020, culminating in the EU Digital Operational Resilience Act (DORA) in 2022.
Initiative/Regulators with an Explicit Encryption Mandate (not an exhaustive list)
- Digital Operational Resilience Act (EU): Mandates encryption for data in transit and at rest ‘where appropriate’ to ensure confidentiality and integrity.
- Monetary Authority of Singapore (MAS): MAS Technology Risk Management guidelines require encryption for external/public networks and secure communication.
- Reserve Bank of India (RBI): Mandates encryption for sensitive data in transit and secure cryptographic key management.
- Hong Kong Monetary Authority (HKMA): Requires secure encryption protocols for critical data exchanges, including trading links.
- Securities and Exchange Board of India (SEBI): Mandates secure connections for trading systems, implying encryption.
Initiative/Regulators with an Indirect/Implicit Encryption Mandate (not an exhaustive list)
- Bank of England (BoE) / PRA / FCA (UK): Requires protecting critical services and secure data transmission, implying encryption.
- Federal Reserve / OCC / FDIC (USA): Emphasizes secure communication. Encryption required for sensitive data in FFIEC IT Examination Handbook (which supports these initiatives).
- Australian Prudential Regulation Authority (APRA): Encryption considered a baseline measure under APRA’s CPS 234 (Information Security).
- Basel Committee on Banking Supervision (BCBS): Recommends encryption as a best practice for protecting sensitive data.
- MiFID II (EU): Focuses on secure IT systems; encryption implied as part of robust system requirements.
Different approaches to meeting these requirements
The Korea Exchange were one of the early adopters of encryption technology in its trading systems. Their EXTURE Plus trading system (launched in 2014) incorporated data encryption to protect data integrity and confidentiality.
Korea Exchange solved the issue in a way which still made analysis of financial timings possible. They did this by putting in place a messaging protocol which encrypted the exchange of credentials and encrypted the transaction details, but kept in plaintext fields that are useful for latency monitoring (message type, a unique message identifier, a member identifier, and the transaction time). This allowed a lot of latency monitoring to remain in place.
Deutsche Börse have followed suit, mandating encryption from October 2023. However, they allow market participants who are located in the Deutsche Börse colo to only require password encryption (which adds a latency penalty at logon time). All other market participants must use payload encryption, which encrypts the whole connection. Importantly for observability, unlike the Korea Exchange’s implementation, Deutsche Börse’s implementation of payload encryption encrypts all parts of the message, making it difficult for clients outside of their colo from analysing their wire data in order to understand their performance.
Other implementations of encryption are expected to be more likely to follow the Deutsche Börse implementation than the Korea Exchange one, because this is easier to implement using standard tools like stunnel, and does not change the underlying trading interface message definition (typically a more expensive change for clients).
Whether the regulators in other territories will be comfortable that local colo traffic remains for the most part unencrypted remains to be seen, and the appetite of exchanges to offer different types of encryption for different types of market participant will likely vary depending on how much of a presence high-frequency trading firms have at the exchange. Crypto exchanges in particular have tended to mandate blanket encryption for all connections, as this is more of a typical ‘internet’ design pattern that suited their early retail investor audience who were likely connecting to the exchange over untrusted networks like the public internet.
The Beeks Solution: Bridging Encryption and Observability
As encryption of order connections gathers pace, trading firms will need to review their observability options and ensure they are still fit for purpose. Beeks offers a number of solutions in this area, and we also work with exchanges to address the visibility blindspot that encryption leaves.
Server-Side Observability of Encrypted Traffic (likely suitable for Exchanges, Trading Platform Providers, or Liquidity Providers)
In this example, Beeks can process session keys provided by the SSL server for server-side decryption:
- How It Works: Network packets are mirrored to the Beeks Analytics appliance from the client-side interface of an Application Controller. Decryption keys are provided to process encrypted streams.
By combining decryption and decoding, firms can monitor performance across encrypted and decrypted streams seamlessly. Clients can, if they prefer, use their packet aggregation network to perform the decryption functionality, if their aggregation layer supports this and they have other (e.g. security) tools.
Client-Side Observability of Encrypted Traffic (likely suitable for Exchanges, Trading Platform Providers, or Liquidity Providers)
- How It Works: This assumes that the client can use a different application to perform the encryption (an SSL Proxy) from their Client Application which is sending the messages.
The server which the SSL Proxy is running on can be fitted with an FPGA card which can mirror the packets to Beeks Analytics. The FPGA card enables minimally intrusive monitoring by securely tapping into application-level data streams without disrupting performance. By using an FPGA card rather than a software-agent based approach, you minimise the latency penalty of having an encrypted connection to your venues and counterparties.
Beeks role in supporting you with your traffic visibility challenges
Where exchanges mandate that you need to be in their colo to have full observability of your traffic to them, Beeks can provide the infrastructure necessary to host your trading systems, either directly to you or (in an increasing number of cases) in partnership with the exchanges using our Exchange Cloud offering.
And if you still require full visibility of encrypted traffic, Beeks has shown that encryption and observability no longer need to be at odds. Beeks’ innovative solutions ensure firms can protect their trading data while maintaining critical insights.
Ready to trade smarter and stay secure? Contact us to learn how Beeks can transform your trading infrastructure.
Further reading
- The 2021 TLS telemetry report from F5 may be a little out of date but still provides an excellent overview of the difference between encryption protocols, cipher suites and ciphers.
- More on the Bangladesh Bank heist and its ensuing money laundering activities in the Lazarus Project podcast (also available on Spotify, Apple, and other podcast platforms)
-Deutsche Börse encryption official circular, FAQ, customer focus call write-up
