One of Cryptonomic’s objectives is to make blockchain technology accessible to a broader audience. While documentation, tutorials, and functional examples are helpful in exposing more people to what’s possible, we also wanted to deliver a set of APIs in familiar languages on a well-understood medium.
More than wrapping new concepts in conventional technology, Conseil libraries offer a highly performant view of chain data that easily lends itself to the sort of reporting transaction systems are expected and often required to provide in business application context. We are using these features in Tezori/Galleon to show account balances quickly. We can imagine a point-of-sale system that can integrate with a procurement system that is able to report on and re-order popular products. It would be applicable too, in the context of enterprise finance where bank of any size are required to provide regulatory reports.
None of this is magic of course. At a high level, we have written several processes in Scala — a capable, JVM-hosted functional language. This code swirls around a PostgreSQL database. On one side it collects, parses, and collates chain operations and metadata. On the other end this information is being made available through web services with a flexible query interface that allows for aggregation and transformation, not just filtering. While it is certainly possible to build yet another block explorer with these libraries (and we have!), the more interesting applications lie in analytics.
Blockchains are often referred to as databases, though by definition, and due to cryptographic requirements, they are ledgers — linked lists if you will. Conseil allows users to query blockchains as if they were relational databases. That’s accomplished by caching chain data in, well, a relational database. This cache eliminates the need to traverse the chain each time some information is required — an adventure that takes an increasing amount of time with every new block added on.
The Conseil data model closely mirrors that of the Tezos chain entities. Eventually, if the need arises, we expect to create task-optimized transformations of this data. This is likely to be necessary as the chain size increases. We may eventually opt to migrate to a columnar store as well. These decisions can be made later since the software architecture of the platform is modular. Conseil is comprised of two major components: Lorre, and, well, Conseil. The former is a chain monitor that picks up new blocks as they appear, collects related data like the accounts touched by the operations in the block, and chain metadata related to the low-level details of how Tezos specifically works. To interface with the database we leveraged an FRM library — Slick. Functional purity is one of our technical objectives, though not at any cost. This lack of tight coupling to the database allows for simplified migration between data stores. At the other end, we have Conseil, a web service front-end written using Akka. Conseil and Lorre aren’t explicitly aware of each other, but they share the data source. The decoupled, stateless design avoids sequence dependencies and the necessity for synchronization.
The above discussion glossed over two important design decisions that warrant an explanation. Slick is a Functional Relational Mapper that provides a more natural database interface in Scala. JDBC is certainly an option, all this code ultimately runs in the JVM but Slick allows for more coherent functional code. There is no need to drop to OO programming just to connect to a database. This library is sufficiently mature that it is able to provide access to a database with minimal configuration in a functionally consistent way. Next, the Akka library is an implementation of the Actor Model. It provides asynchronous execution without locks. It is a framework for writing highly scalable, decoupled code that encapsulates system functionality into easy to understand components. The Actor Model has been used for a long time provide system level scalability. For example, we’re used to seeing a solution architecture including multiple web servers sitting behind multiple caching servers all talking to the same database, perhaps through an additional caching layer. Akka gives us this model at application component level.
We are excited about blockchains being the plumbing of the new financial system. In a later article we’ll discuss the motivation behind ConseilJS and explain some of the design choices we made.