Running an Ethereum Cardstack Hub Application

How to wrap a smart contract in a RESTful API

We are currently working on a documentation library that will soon be viewable at Cardstack.com. In the meantime, we will be publishing a series of technical blog posts that walk new developers through using the Cardstack framework for the first time.

In a previous post I discussed how the Cardstack Hub integrates with Ethereum. In today’s post, I’m going to show you how to run a Cardstack Hub against your smart contract. For those new to my blog, I’m an Ethereum developer at Cardstack, and I routinely post about fun challenges we are solving.

At Cardstack we are building an application framework for decentralized applications (dApps) that puts user experience first. One of the foundational aspects of our framework is our ability to expose on-chain information to dApps in an easy-to-consume manner that allows developers to query the blockchain without any special Ethereum libraries (like web3.js). Let’s set up a Cardstack Hub to serve your smart contract as a RESTful service.

The sample Cardstack application in this blog post is available on GitHub if you want to see the full source code for this application.

Create an Ember Application

The Cardstack Hub is ideally suited to be consumed by an Ember.js application (albeit, it doesn’t have to be). Since Ember.js applications are easy to setup and have great tooling, let’s start there.

Creating an Ember.js application is really simple. Let’s start by installing the yarn package manager. The simplest way to install yarn, if you haven’t already, is to execute:

curl -o- -L https://yarnpkg.com/install.sh | bash

Now let’s install the Ember.js tooling by executing the following from any directory:

yarn global add ember-cli

This will allow the ember executable to be available from your command line.

Next let’s create the Cardstack Application. We’ll name it: smart-contract-example. Execute the following command:

ember new smart-contract-example

This will create a directory called smart-contract-example/ and setup a vanilla Ember.js application. Try out your Ember.js application by changing to the application directory that was just created and starting the development server:

cd smart-contract-example
ember serve

You should see something that look like the following:

Output of the ember serve command

Your Ember.js application is now running on your machine. You can view it at http://localhost:4200.

A running Ember.js application

The Ember.js application that we just set up can be used to build a user interface that then consumes API from the Cardstack Hub. We have some really powerful tools for building immersive Cardstack user interfaces that we’ll blog about more in the future. For now, we’ll just focus on the API that the Cardstack application will consume.

Installing Cardstack Hub

OK, now that you have a vanilla Ember.js application running, let’s install and configure a Cardstack Hub for the Ember.js application.

First, let’s install the @cardstack/hub Ember.js add-on. From your sample-smart-contract project folder execute the following:

ember install @cardstack/hub

We also want to add some additional npm packages. Using yarn, let’s add the following packages to our project:

yarn add @cardstack/ethereum @cardstack/jsonapi @cardstack/test-support @cardstack/eslint-config eslint-plugin-node

To make life easier inside of whatever code editor you are using, we have bundled an eslint configuration that is inclusive of Ember.js’s eslint rules as well as rules for the Cardstack Hub. Edit the .eslintrc.js file and use the following configuration instead of the one that was created for you:

.eslintrc.js file

Now let’s configure an Ethereum data source. We happen to have a CARD token contract deployed to Rinkeby at the address 0x013c05c37d24d96e4cc23e5d0efcd2aa13d81d7c.

(If you are curious, the source code for the CARD token can be found on GitHub.)

In the cardstack/ directory within our project, let’s create a new subdirectory for our data sources called data-sources. And within that, we will create a data source configuration file for the Rinkeby CARD token called card-token.js.

mkdir cardstack/data-sources
touch cardstack/data-sources/card-token.js

Now, let’s edit the cardstack/data-sources/card-token.js file. Within the @cardstack/test-support package is a very handy module: JSONAPIFactory. We can use this module to easily construct the objects and relationships that the Cardstack Hub consumes, as well as to emit models in an order that respects relationships that you have defined.

card-token.js data source configuration file

On line 2, we reference an ABI file for the smart contract. For our CARD token, you can find the ABI as a JavaScript file here.

Let’s place this file at cardstack/token-abi.js.

You’ll notice on line 9 we are referring to a URL where we can access the WebSocket interface for an Ethereum Geth node pointing to the Rinkeby network.

We specify the address of the CARD token contract available on the Rinkeby network on line 13, as well as the smart contract events that we are interested in on lines 15–17.

In my previous blog post The Cardstack Ethereum Plugin: Technical Overview, we covered how the Cardstack Ethereum Plugin uses these events to index your smart contract.

External Dependencies

The Cardstack Hub has a few external dependencies it requires in order to operate correctly. Let’s go through them.

Docker

Docker is not a required dependency, but we highly recommend it. Docker is a great way to bundle services in simple and composable packages. For the examples below, we will assume that you have Docker installed. You can find installation instructions for Docker here.

Postgres

The Cardstack Hub uses a PostgreSQL database to persist the content index. Additionally, the PostgreSQL DB is used to orchestrate job queues used for indexing.

A PostgreSQL DB can also serve as a data source for content. But for this example, we’ll just be using Ethereum as our data source. (Stay tuned for future blog posts where we explore other data sources.)

The Cardstack Hub will automatically try to attach to the default PostgreSQL DB, postgres://postgres@localhost:5432/postgres; however, you can specify a PostgreSQL DB using the standard PostgreSQL environment variables (PGHOST, PGPORT, PGUSER, etc), or you can specify a PostgreSQL URL to your database using the environment variable DB_URL.

For our example, let’s just use the default PostgreSQL DB. We have provided a handy Docker container that you can use to easily spin up a PostgreSQL database:

docker run -d -p 5432:5432 --rm cardstack/pg-test

This will start a PostgreSQL database on port 5432 on your local system.

Geth

For the purpose of this blog post, we’ll be setting up a Cardstack application that will use an Ethereum smart contract as its data source. The Cardstack Hub uses Geth as the interface to Ethereum. Geth is a Go implementation of an Ethereum client, and probably the most popular Ethereum client used today.

The simplest way to setup Geth is to use Docker. To run Geth, we’ll first setup a folder on your local machine to act as a filesystem volume for Docker. This allows your downloaded Ethereum blocks to persist between Docker container restarts.

mkdir ~/ethereum

Now we can start Geth using Docker, and tell Docker that we want to use ~/ethereum as the filesystem volume.

In this example, we’ll be attaching to the Rinkeby test network. Note that when you provide the path to your volume, you’ll need to use the absolute path. On my machine, ~/ethereum resolves to /Users/hassan/ethereum.

docker run -d --name ethereum-node -v /Users/hassan/ethereum:/root -p 8545:8545 -p 8546:8546 -p 30303:30303 ethereum/client-go:stable --rinkeby --rpc --rpcapi eth,net,web3 --rpcaddr 0.0.0.0 --ws --wsaddr 0.0.0.0 --wsorigins '*' --wsapi eth,net,web3 --cache 4096

One item to point out: we are turning on both the RPC interface on port 8545 and the WebSocket interface on port 8546. The Cardstack Hub uses the WebSocket interface of Geth in order to index content from Ethereum.

Geth will now start downloading blocks from the Rinkeby network. Depending on your internet connection this may take a couple hours. You can then use the following command to keep track of the downloading process:

docker logs -f ethereum-node

Until Geth has caught up to the latest block, it won’t respond to clients. So you’ll need to wait for Geth to download its blocks. From the Geth logs you can keep track of the block number that is being downloaded and compare it to the latest block on Rinkeby here. Once Geth has caught up, it will be ready to use.

In order to make life easier for AWS deployments, we have actually created a Terraform module to deploy a Geth node into AWS. If you are interested, you can find the Terraform module here.

Node.js

The Cardstack Hub server is hosted by Node.js, a JavaScript-based server. The Cardstack Hub will automatically run on the Node.js server that hosts your Ember.js application when you execute ember serve. Additionally, if you prefer your Cardstack Hub to run on a separate Node.js server, you can start your Cardstack Hub with the command:

ember hub:start

This will start your Cardstack Hub on port 3000. You can also supply a PORT environment variable to run the Cardstack Hub on a different port. In this particular mode, you can inform the Ember.js application where to find the Cardstack Hub using the HUB_URL environment variable, which is set to the URL of your Cardstack Hub server.

For our purposes, it is easiest to allow the Cardstack Hub to use the Node.js server that is hosting your Ember.js application. In this case, we can start the Cardstack Hub using the same command that we use to start our vanilla Ember.js application:

ember serve

Now when we start our application we can see the Cardstack Hub starting up:

Output of ember serve command while the Cardstack Hub starts

Running the Cardstack Hub

Once the Cardstack Hub starts up, it will discover the data sources that you have configured in the cardstack/ folder of your project and begin indexing those data sources.

The Ethereum data source takes a couple minutes to index all the Ethereum events, but the contract content itself should be indexed within seconds of the Cardstack Hub starting. Let’s start exploring our contract!

The Rinkeby CARD token contract is being served at the endpoint: http://localhost:3000/api/card-tokens/0x013c05c37d24d96e4cc23e5d0efcd2aa13d81d7c

The RESTful response for the Rinkeby CARD token

In my previous blog post The Cardstack Ethereum Plugin: Technical Overview, we discussed how the Cardstack Hub indexes Ethereum data sources. The read-only functions of the smart contract that do not take a parameter are available as attributes of the HTTP GET response above using the prefix card-token and the dasherized name of the smart contract function.

As the configured Ethereum events are emitted every ten minutes from cardstack/card-token.js, the attributes of the smart contract above are updated in the Cardstack index. Additionally, the meta property provides context of when the smart contract was last indexed with the blockheight property.

Another one of the content types that we are indexing, specified in cardstack/card-token.js, is card-token-balance-ofs. The card-token-balance-ofs content type represents the various token balances in our smart contract (this corresponds to the ERC20 balanceOf() smart contract function). We can get a listing of all the token addresses by issuing this GET: http://localhost:3000/api/card-token-balance-ofs

The RESTful response for the Rinkeby CARD token balances

This response will return all the token balances using a default page size of 10 token balances per page (the page size can be altered using query parameters).

Also, you’ll note that the meta property in this response indicates how many total records exist, as well as provides a link to get the next page of token balances.

The address of the token holder can be found in the attributes.mapping-number-value property within each card-token-balance-ofs resource in the response above. (Note that we use a JavaScript string type to represent this number, as Ethereum numbers can be much much larger.)

We can issue the request http://localhost:3000/api/card-token-balance-ofs/0x88e1d504fd6551a7b7b19e1aa6881de1a9f18ca7 to look at the balance for a specific token holder, whose address is 0x88e1d504fd6551a7b7b19e1aa6881de1a9f18ca7

The RESTful response for the Rinkeby CARD token balance of token holder 0x88e1d504fd6551a7b7b19e1aa6881de1a9f18ca7

Additionally, within the card-token-balance-of resource is an array of Transfer events related to the ledger address. The Transfer event is an ERC20 event that is emitted when there is a token transfer that occurs in the smart contract. This this is the card-token-transfer-events content type.

We can use a query parameter to search for all the card-token-transfer-events where the token holder is a recipient of a token transfer by issuing a request GET http://localhost:3000/api/card-token-transfer-events?filter[transfer-event-to]=0x88e1D504Fd6551A7B7b19e1Aa6881de1A9F18ca7

The RESTful response for the Rinkeby CARD token Transfer events where the token holder 0x88e1d504fd6551a7b7b19e1aa6881de1a9f18ca7 is a recipient of a token transfer

The card-token-transfer-events content type describes all the event arguments in the Ethereum event it represents. In this case, that includes the from, to, and value event arguments for the ERC20 Transfer event. Additionally, the block number as well as the transaction ID that this event was emitted from is included in the response body for this resource.

Tools for the Next Generation

In this blog post, we have demonstrated how to wrap an Ethereum smart contract in a RESTful API. Applications that consume these RESTful API’s do not need any specialized libraries in order to communicate with your Ethereum smart contract.

The Cardstack Hub does all the heavy lifting in regards to monitoring your smart contract for changes, decomposing the smart contract state into interrelated models, establishing a searchable index for your contract’s models, and serving those models. This allows the application developer to focus on the domain-specific logic around their contract, instead of trying to build a boilerplate framework that all smart contracts will need to leverage.

We believe that the Cardstack Hub will empower the next generation of developers to build applications that can blend on-chain and off-chain data sources in a simple and easy-to-use manner.

Taking It Further

This example just scratches the surface of what is possible with a Cardstack application.

A very exciting new feature that we’ve recently added to the Cardstack Hub is the ability to create server-side computed properties against your data sources. With server-side computed properties, you can augment your data sources with business logic. For example, imagine an Auth0 identity data source that uses an Ethereum smart contract data source to describe the permissions a user has upon services offered from an application.

I’m really looking forward to creating a new blog post that dives into the possibilities of server-side computed properties and how to create your own with the Cardstack Hub. (In the meantime, I’ll leave you with this example of using server-side computed properties in the Cardstack Ethereum data source from our unit tests).

Additionally, you may have noticed in this example we didn’t really dive into the user interface of the Ember.js application that we created. We’re building a really compelling user experience framework that lives on top of the Ember.js framework that really merits its own blog post (or series of blog posts, really).

Stay tuned!

Read More

• The Cardstack Ethereum Plugin: Technical Overview by Senior Blockchain Developer Hassan Abdel-Rahman
• Growing a Healthy Software Ecosystem by Lead Developer Ed Faulkner
• How the Cardstack Framework Powers Decentralized Applications
• The Cardstack White Paper

Get Involved

Join the discussion about Cardstack on our official Telegram channel: https://t.me/cardstack.

Are you a developer? Join our Slack channel: https://join.cardstack.com

Important Reminders

• We will never, under any circumstances, solicit funds from you via email, private message, or social media.
• If you are in doubt or notice any suspicious activity, please message the admins in our official Telegram group: https://t.me/cardstack.