What is the Cardstack Hub?
A layman’s introduction
The Cardstack Hub is one of the most important parts of the Cardstack architecture — it lets applications easily handle multiple data sources, whether in the cloud, on the blockchain, or somewhere else.
The following blog post is a quick, layman’s introduction to what the Hub is and how it works, based on a loose transcription of the above video.
Note to Developers
We are working on documentation for the Hub. In the meantime, you can learn more about it by reading our whitepaper, or by visiting our GitHub page, where the Cardstack Hub is fully open-sourced.
It Starts with Cards
A key aspect of the Cardstack architecture is the idea that we represent web applications as “cards” — extremely rich functionality packaged up into component form. We want all these different components from different sources and different authors to coexist in your web browser, which is a great general application runtime.
The challenge is getting these apps to talk easily to many different data sources on both the centralized and decentralized Internet.
Dealing with Many Data Sources
It’s likely that your digital life is located across many different data sources. Some data might be in a traditional SQL database like PostgreSQL. Some data is on the Ethereum blockchain. Or Git. Or a fully hosted data source.
People suffer because their data is trapped in all these different siloes. Your stuff in your Google Calendar doesn’t know anything about your stuff in your accounting database, and your Quickbooks, and it’s not easy to link that up with some smart contract in Ethereum.
How do we bring everything together? You, as the owner of your data, have to be the one running your own personal search engine. And that’s what the Cardstack Hub is designed to be.
The Cardstack Hub Unifies Your Digital Life
The idea is that you can index all the content that’s relevant to you — out of all your data sources — into your own Hub. Technically speaking, the Hub is a Node.js-based server. And within the Hub is ElasticSearch or PostgreSQL indexing — a powerful search engine that lets you denormalize data out of bunch of canonical sources into a very fast searchable tier.
Once you index content into your Hub, all the interactivity happening on your front end card applications has a really fast standard interface to all your data, through a standard JSON:API spec, or WebSockets for real-time push and sync. And that data will be cross-referenced and holistically connected even though it comes from many different sources.
Because all of that takes place between the front end and the Hub, and it means that your data sources can have radically different speeds, radically different uptimes. Which is important when dealing with slower data sources like blockchains.
To make this possible, you need a classic adapter pattern. We’ve built very good plugin capabilities in the Hub, so that you can write an adapter for the Hub that knows how to talk to PostgreSQL. Just as you can have an adapter that knows how to talk to Ethereum, and it can be packaged up with its own microservices so that for example, you could say, “I would like to add Ethereum to my Hub,” and the plugin would spin up the necessary microservices to host Geth.
Cardstack Plugins
We have a well-typed plugin system, as opposed to a let-plugins-do-whatever-they-want kind of system.
By making them really well-typed, every plugin provides features, and there’s an enumerated set of possible kinds of features. Some of them are very general and powerful, like middleware, but others can be much more focused.
For example, to implement a data source, you’re going to need some combination of an indexer feature, a writer feature, and a searcher feature.
As it turns out, these three kind of well-typed plugins are enough to do vastly different backends:
Indexer
An indexer is something that can incrementally index content out of elsewhere into your own search index. That’s the most powerful way to use data because now you have a local fast cache of everything.
This gives us a lot of power to apply things like authorization rules. Whatever rules that you have for that data can be implemented at indexing time. So even if you have a very rich set of authorization rules, we’re able to just denormalize all the facts out into your search index and have a very search engine-like experience on the front end, where you’re not limited by massive table joins, authorization checks, and so on.
Writer
The writer plugin does what it sounds like: it writes to the data source.
It’s important to realize that the authoritative home of data is the backend. So data that you care about should be on your backends that you own and control.
Data that you’re trying to integrate with still lives where it’s supposed to live, and the Hub Tier really is a caching tier. So writer plugins don’t write to the cache.
Instead, we go full round-trip: writer plugins go all the way through the Hub, write to the authoritative data source, and then the indexers pull that content back in. This means you get all the native semantics of the backends.
For example, if a write to Ethereum is causing things to happen — smart contract code runs and changes the state — we’re going to see that reflected in the index.
Or if a PostgreSQL database is being touched by other code somewhere else, or it’s got triggers, that’s going to matter.
Searcher
The searcher lets us handle requests through the Hub. We can go do deeper searches for things that we don’t have in the search index locally.
For example, maybe you’ve decided in your application you’re going to outsource identity management to somebody else. Maybe you’re going to let users login with GitHub profiles.
In that case, you want to still represent a user in your application. You need that model that represents the user.
You don’t necessarily want to index every GitHub user into your Hub, so in that case, you would just use a searcher, so when you want to refer to a certain user, that request wouldn’t hit your own search index, it would go out to the end.
Full-Stack Decentralized Applications
The grand vision here is you can package up full sets of functionality, all the way up and down this stack.
For example, as a developer, if you wanted to build a dApp that uses Ethereum as a data source, you could build front end cards that know how to reflect the data that’s part of your application. You would also provide the plugins that would give the cards the server side code they need in their Hub to talk to the Data Tier.
And of course, maybe there’s work to do in the Data Tier itself. Maybe there’s a smart contract to be deployed. Maybe there’s a PostgreSQL database that needs to be created. Maybe there’s a Git repo to be created.
The point is, you can package up all the tiers as a reusable piece.
Once you do, you can have a whole open ecosystem using this stuff, all at once, together.
Toward Multiple Blockchains
The first blockchain plugin we’ve written allows the Hub to talk to Ethereum/Geth. But since the Cardstack architecture supports multiple simultaneous data sources, and provides a well-structured plugin API that ensures they are all equally first-class, there’s no limit to how many plugins — and how many blockchains and dApps — can be added.
That means Cardstack will make it simpler than ever to create apps that can easily index, search, and write to decentralized data sources. To put it another way, the Hub means any blockchain can interoperate with other blockchains, dApps, and existing data sources.
Final Thoughts
Something great about the Cardstack architecture is that the individual pieces are not radically different than what experienced teams are building today. The patterns come out of hard-won experience from having built this kind of application many, many times over.
We’ve already proven our approach with our minimum viable product, the Cardstack Token Generation Event web app, which was fully powered by the Cardstack Hub. Our upcoming app, Cardspace, will demonstrate the Hub’s abilities when paired with our composable UI/UX.
The goal is to make the Cardstack Framework the common developer toolkit for future decentralized apps — and to start shipping the kind of software that can fully encompass a user’s digital life.
Read More
- Growing a Healthy Software Ecosystem by Lead Developer Ed Faulkner
- Running an Ethereum Cardstack Hub Application by Senior Blockchain Developer Hassan Abdel-Rahman
- How the Cardstack Framework Powers Decentralized Applications
- The Cardstack White Paper
Get Involved
Join the discussion about Cardstack on our official Telegram channel: https://t.me/cardstack.
Important Reminders
- We will never, under any circumstances, solicit funds from you via email, private message, or social media.
- If you are in doubt or notice any suspicious activity, please message the admins in our official Telegram group: https://t.me/cardstack.
