Amazon QLDB: Bringing the power of blockchain to your databases

Co-Author: Ritesh Menon
LinkedIn: Mayank Vora, Ritesh Menon

Ledgers are typically used to record the history of an organization’s economic and financial activity. Many organizations create applications with ledger-like functionality because they want to maintain an accurate history of their data. For example, they might want to trace the movement of an item in a supply chain network or track the history of credits and debits in banking transactions. Custom audit tables or audit trails created in relational databases are frequently used to implement ledger applications. Furthermore, custom audit tables and trails require custom development, which demands more time and financial investment. Furthermore, human errors are common in custom development.

In recent years, we have seen widespread adoption of Distributed Ledger Technology (DLT) such as blockchain for record keeping. These distributed ledgers are decentralized in the sense that there is no central administrator or database, and authentication is established through a peer-to-peer network using consensus algorithms. The records are immutable and cryptographically verifiable through the use of hash algorithms such as SHA-256, SHA-512, and others.

The diagram above depicts a distributed ledger, which is a peer-to-peer network of n nodes. Network nodes are self-contained computers that synchronize transactions in distributed ledgers.
The consensus protocol ensures that every new block added to the distributed network is the one and only version of the truth that all nodes in the network agree on.
Any change or addition made to the ledger is immediately reflected and copied to all participants once trust is established.

Distributed ledgers are broadly of two types:

1. A public distributed ledger is required when there is a need for an open network that is transparent, distributed, secure, and serves larger communities.

For example, cryptocurrencies (Bitcoin (BTC), and Ethereum (ETH)) are based on public blockchain technology, where transactions are transparent, immutable, and distributed. These transactions are verified by network participants, those who have the required hardware, and are known as “Miners.” Anybody can join a network as a miner and get rewards by verifying the transactions and adding a new block to the blockchain.

A public ledger becomes more secure as it becomes more decentralized and active. As more people use the network, it becomes more difficult for any type of attack to succeed.

2. A private distributed ledger is a good choice when an organization wants to leverage distributed ledger features while restricting access to specific users in a closed distributed network. For example, maintain records of employee salaries, sales, purchases, rent, stationery, insurance, and so on in a private ledger. Hyperledger and Ripple are two popular examples of private blockchains.

Distributed ledgers are effective in certain scenarios, but not in all. Such systems have some drawbacks, as listed below:

• It requires more time for the network to reach a consensus and verify transactions.
• High energy consumption.
• Increases the complexity of the application.

Furthermore, there are some situations in which an organization (such as a financial institution) does not want to share a ledger with any other party and instead prefers to keep track of data on a centralized system.

Some questions to ponder over:

What is Amazon QLDB?

Amazon QLDB is a new class of centralized ledger databases that makes it easier to avoid the time-consuming process of developing your own ledger-like applications.

Its salient features are as follows:

• Immutability & Transparency: Traditional databases (SQL and No-SQL) store data as a table or JSON document model. Data in these traditional databases can be modified by anyone who has access to the database, resulting in data conflicts and manipulations.
  Amazon QLDB includes a journal that stores data in an immutable manner. The journal is append-only, which means that data can only be added to it and cannot be modified or deleted. You can read from the immutable journal to view the change history of the data even if you delete it from your ledger.
• Cryptographically Verifiable: Because traditional databases lack automatic data encryption, data files are vulnerable to being read or modified directly by hackers. Some of these databases are incompatible with cryptography.
  Amazon QLDB uses encryption to create a brief history of your change. A cryptographic hash function(SHA-256) is used to generate this secure summary, also known as a digest. You can go back in time and verify the accuracy of your data modifications using the digest as evidence of your data’s change history.
• Highly Scalable: Amazon QLDB is a serverless database that provides seamless, automatic scaling to meet an application’s needs, eliminating the need to provision capacity or configure read and write limits. Amazon QLDB makes Amazon CloudWatch metrics available for the ledgers. We can view key operational metrics for read and write IOs using QLDB.
• Audit Logs & History: Traditional databases do not keep track of document history and only store the document’s current state. As a result, these databases are unable to provide audit trials out of the box. Many SQL Server auditing solutions available on the market lack the ability to ensure tamper-proof auditing.
  QLDB can provide audit trails with no additional configuration. We can get transaction history details by querying for a summary of historical changes using Amazon QLDB.
• PartiQL Support: It uses an open-source, SQL-like query language, PartiQL which allows us to query, manage, and update data with familiar SQL operators. Amazon Ion is the document-oriented data model used by QLDB. Amazon Ion is a superset of JSON. Ion is an open-source, document-based data format that gives the flexibility of storing and processing structured, semistructured, and nested data.
• Streaming Capability: Amazon QLDB’s streaming capability enables Amazon Kinesis Data Streams to deliver near-real-time updates to data stored in QLDB. This allows us to react quickly to new events (such as a change in account balance in a banking ledger application) and integrate with downstream services easily (e.g., AWS Lambda, Amazon Redshift, Amazon Elasticsearch).
• Easy to use: Amazon QLDB stores data using a document-oriented data model, which provides the flexibility to store structured and semi-structured data.

A Quick Walkthrough of QLDB Features

Amazon QLDB Walkthrough

We have compiled below walkthrough video on QLDB for you to get a quick understanding of the following concepts:

• Getting started with QLDB
• Create a ledger and perform basic CRUD operations.
• How to connect QLDB programmatically with a Node JS application?
• How does QLDB store the data in an immutable way?
• How can we see the transaction history?
• How can these records be cryptographically verified?

Amazon QLDB Demo

Which organizations have adopted QLDB?

For an exhaustive list of organizations that have already successfully used QLDB, check out this link.

Are there any limitations to QLDB?

Every technology will have limitations, and so does the QLDB. The following is a list of a few of the limitations of QLDB:

• Is it possible to take a snapshot or backup of the ledger? — At this time, Amazon QLDB does not support backup and restore. The ability to export to S3 is currently available. We can export the contents of the QLDB ledger to S3 using this feature.
• Is it possible to restore the ledger to a specific point in time? — As of today, Amazon QLDB does not support a point-in-time restore feature.
• Is it possible to sort the data? — There is no support for sorting using the ORDER BY clause. Hence, there are no guarantees for the ordering of query results.
• Is pagination supported by QLDB? -Pagination is not supported using the LIMIT clause.

QLDB is still in its infancy but holds a lot of promise. The aforementioned limitations can be expected to be resolved in future releases.

Now I think you will have a similar question in your mind:

Let this simple workflow guide you in making a decision…

Conclusion

Blockchain and other distributed ledger technologies are gaining popularity because they keep records in an immutable, secure, and cryptographically verifiable manner. As a result, traditional databases may make a way in favor of more robust blockchain-based databases, which store data in a centralized way for faster operations, scalability, and sustainability. Ledger databases, such as Amazon QLDB or ImmuDB, appear to be viable options that could see widespread adoption in the future.