Developer Blog: Hub Improvements
Combining on- and off-chain data
Today, we describe updates we’ve made to the Cardstack Hub since the successful close of our TGE.
What is the Cardstack Hub?
The Cardstack Hub is our open-source, Node.js-based data orchestration layer that lets apps easily talk to on- and off-chain data sources. This means developers can create user experiences that reflect a user’s complete digital life, across the cloud and decentralized Internet.
The Hub works by indexing data using plugins. That data is then brought forward to applications using APIs/Websockets. For example, during our TGE, the Hub indexed realtime on-chain fundraiser data using an Ethereum/Geth plugin and served it to our Dashboard.
Optimizing Indexing
The design of the Cardstack Hub excels by allowing for very simple ways to express relationships between completely different things — for example, what if you want an on-chain smart contract to take some sort of action when a change appears in a totally separate PostgreSQL Database?
To do that, we have to interrogate the data sources and express the relationship with a server side computed property — which the Hub can then cache and memoize.
We have to make sure that the data indexed by the Hub is totally consistent, so that these relationships can be introspected by applications with total confidence.
PostgreSQL vs ElasticSearch
In our initial implementation of the Hub, we used ElasticSearch as our indexer. ElasticSearch is a great non-realtime, low-consistency search index, but we need to read our own writes in realtime and maintain consistency across documents.
Now, we’re moving to the next level by using PostgreSQL — which should improve both the latency and the consistency of indexing. PostgreSQL is well-known for classic relational use cases, but less commonly recognized is that it also makes a fine denormalized document store, pulling in some of the best ideas from the NoSQL world.
Prior to Cardstack, there has not been a way for on- and off-chain data sources to talk to each other this easily. Our approach accomplishes this elegantly, and provides developers with good semantics.
Improving the Hub’s approach to bulk indexing
Currently, large indexing events slow down the Hub’s response time for user requests (for example, logging in to an app).
The improvements we are implementing will allow bulk indexing events to be completed in the background, so that user requests can be processed in the meantime.
Improving the Hub’s connection to Geth
We are adding logic to ensure the Hub automatically retries connecting to Geth if a connection is lost. We are adjusting our Terraform scripts to provision optimal infrastructure in AWS for our Geth nodes.
Read More
- The Cardstack Ethereum Plugin: Technical Overview by Senior Blockchain Developer Hassan Abdel-Rahman
- Growing a Healthy Software Ecosystem by Lead Developer Ed Faulkner
- How the Cardstack Framework Powers Decentralized Applications
- The Cardstack White Paper
Get Involved
Join the discussion about Cardstack on our official Telegram channel: https://t.me/cardstack.
Important Reminders
- We will never, under any circumstances, solicit funds from you via email, private message, or social media.
- If you are in doubt or notice any suspicious activity, please message the admins in our official Telegram group: https://t.me/cardstack.
