Announcement: Matching Engine Finished
We’re happy to announce that we finished the development of the Matching Engine. The Matching Engine is a critical component and we’re quite proud with the result since we overcame major challenges and reached our goal of:
- high performance,
- reliability,
- maintainability,
- high availability and
- disaster recovery.
Technical Details
High Performance, Reliability and Maintainability
The Matching Engine takes orders as an input, matches them and produces trades as an output. This first version implements a price-time priority matching algorithm (FIFO): orders are first ranked according to their price; orders of the same price are then ranked depending on when they were entered.
http://www.cmegroup.com/education/matching-algorithm-overview.html
The Matching Engine was developed in the latest version of Java, Java 10, a very reliable, maintainable and well tested platform. While Java has the reputation of being less performant than other languages, it comes down to how proficient you are in the use of the language and the JVM. We are very satisfied with the performance results.
For faster and easier integration of the Matching Engine we are using Docker Containers.
https://blog.docker.com/2018/04/improved-docker-container-integration-with-java-10/
The core is based on the CQRS principle, using the Axon framework
https://martinfowler.com/bliki/CQRS.html
Our CQRS command bus is based on the famous LMAX Disruptor which is basically a ring buffer avoiding blocking and contention in multithreaded environments
https://martinfowler.com/articles/lmax.html
The business logic is single threaded. For performance reasons, all live states (order books) are held in memory. Everything is non-blocking, asynchronous and event-driven.
High Availability & Disaster Recovery
All business events (eg. ‘trade executed’ or ‘price level changed’) are journaled in a clustered database. In case of a JVM crash, the in-memory state is rebuilt by replaying these events from the most recent snapshot.
At least two active replica of the matching engine are created for each tradable pair (eg. 2 identical instances for BTC/USD). They process exactly the same inputs (orders) and create the same outputs (trades) in parallel. If a replica is lagging or crashing, then it is restarted and recovered. Front-end modules always read the stream of trades from the fastest running replica.
Deterministic
The behaviour of the matching engine is absolutely deterministic. Given a starting snapshot and given the same sequence of events, the business logic ends up in exactly the same state and produces exactly the same output. This is beneficial for testing, recovery, diagnostics, analytics and auditability.
Next Steps
Our team started to plug the matching engine with the wallet for which the development is progressing fast. An external team will start to audit the code and proceed penetration test next week. Stay tuned! we will post an update regularly.
Best Regards,
Ouziel Slama, CTO
