Quantum Ledger Database(QLDB) + Serverless

Published in

firstlineoutsourcing

6 min readNov 3, 2021

I bet you can’t generate anything more motivating than this

Write it down, right now! Yes, at the end there will be something like «This is Shit, redo…». But you have already exist Shit!
(c) Andrew Zaikin

My experience shows that every new developer day is hard research. Research the depths of those technologies that yesterday seemed completely understandable. Today, this research led me to a blockchain-based database. We will use it in conjunction with my favorite serverless framework, and most importantly, we will see what tasks this technology solves.

Usually, our company automates processes related to media content or broadcasting. Mostly, DynamoDB or PostgreSQL is enough for such tasks. But it has already been said a lot about these technologies. So I’ll tell you about the task where these capabilities were not enough.

User Guide

Research by Quantum Ledger Database

General Information
QLDB sandbox or «Where can I just try it?»
Creating QLDB resources + Serverless
Working with QLDB table + Serverless
TODO App
TODO App + QLDB Stream
Conclusion

General Information

Let’s start! Imagine you have an application that needs to make financial transactions, keep a log of transactions, maintain a user’s balance, and respond to data changes in the database. Would you use DynamoDB for these tasks? Perhaps! Would you have any difficulty with this? I think so. What are the alternatives? I’ll tell you now.

And now the Quantum Ledger Database enters the scene. Of course, we'll start with the definition from the documentation.

Amazon Quantum Ledger Database (Amazon QLDB) is a fully managed ledger database that provides a transparent, immutable, and cryptographically verifiable transaction log owned by a central trusted authority. You can use Amazon QLDB to track all application data changes, and maintain a complete and verifiable history of changes over time.

As you understand, everything is centralized and stored on Amazon servers. The blockchain mechanism implies decentralization. But we only have a «cryptographically verifiable transaction log». We can still be sure that no one can change the data unnoticed because each transaction stores a part of the previous one.

Long story short. What does QLDB give us?

Transaction log out of the box
The ability to subscribe to all changes
The ability to manipulate the database from code, such as creating and deleting tables

Will this help us in creating our application? Definitely yes!

QLDB sandbox or «Where can I just try it?»

To just try QLDB go here. You can create a test Ledger and do some simple queries.

To work with QLDB, we need to use PartiQL. It is very similar to SQL and if you know it, then there will be no difficulties

PartiQL is a SQL-compatible query language that makes it easy to efficiently query data from relational databases, NoSQL databases, local file system regardless of where or in what format it is stored.

Now, we can go to Query editor, select our ledger, and run a query that creates a table.

CREATE TABLE SomeTable

Once we have a table, we can manipulate the data in it. For example, we will add a new record to it like this.

INSERT INTO SomeTable value {‘id’: ‘123’, ‘data’: ‘some data’};

Other queries will work similarly. Finally, I will show the history of the table. Everything is very simple.

SELECT * FROM history(SomeTable)

If your goal is simply to overview this technology, then this information is enough for you. Next, I will show how to use QLDB together with Serverless.

Creating QLDB resources + Serverless

I will not create a new project, because usually, we in First Line Outsourcing use this skeleton.

If you like to read code from the repository then check here.

Let the code begin! What will you first need when we start working with the new service in Serverless? Cross yourself? Call mom? No. It is necessary to create resources. In the case of QLDB, we need to create Ledger and tables. With Ledger, everything is simple, we create a resource, set the AWS::QLDB::Ledger type and other properties. Ready!

Now the tables. We can create tables using the SDK, for this we have to get our hands dirty and write a little code. Okay, I’ve already written everything, here is a service that can help you create tables and indexes.

But that is not all. We want all resources to be created during deployment and not sometime later, so we need to create a resource that will have a Custom::qldbTable type and an associated lambda. In the lambda, we will put the creation of the tables and indexes we need. It is important that this resource depends on our Ledger to be created after it.

The new resource looks like this:

The lambda wich creating table looks like this:

Ready! Now just start this command in terminal

npm run deploy:dev

Working with QLDB table + Serverless

After creating the resources, we need to understand how to work with them. This is where the commands we used in the first chapter will help us. We will use these commands through the SDK to perform simple CRUD operations on our tables. I went a little further and wrote a service that we can extend, pass an interface to it, and work with the model, it looks like this:

Hope this is useful to you. Here is a usage example:

For example, this is how we can add some kind of new record to SampleModel:

TODO App

It’s time to write some APIs using everything we already have. We will create AccountBalance table and write queries that will add a new record about the user’s balance, update the balance itself, increase or decrease, as well as return its value by ID. Let’s get started!

AccountBalance model looks like this:

Now you need to initialize the functions:

Also need to write the code:

TODO App + QLDB Stream

Finally, I’ll show you how you can subscribe to changes in your tables. We already have an AccountBalance table. For example, we want to react to changes in the user’s balance. What do we need for this? Create a stream!

Also, we need a lambda that will listen to all changes. Here it is

Here you can read about the format in which the data is received from the stream, but I’ll just show you how to get it.

We only need to respond to balance updates, so the data will be filtered by that value in the query string

UPDATE AccountBalance as item SET item.balance =? WHERE item.id =?

The lambda code:

Conclusion

Today we all learned a little more. It is up to you to decide whether this technology deserves attention, but if it does, then I hope my code will help you!

For this project, I used the skeleton, which we in First Line Outsourcing use daily.

You can find here the repo with all code

If I missed something, let me know in the comments. Let’s share the experience and make each other more experienced.