Muttniks: An open source dapp to show you how we built Astro Ledger
Introduction
Launching a dapp may not be as hard as launching a rocket… but it’s no moonwalk, either. A dapp, portmanteau of “decentralized app,” is any app that runs on decentralized infrastructure, like on Ethereum. For your coding pleasure, our team is happy to present Muttniks, a friendly Ethereum space doggo “kernel” (or kennel) that you can build from the comfort of your spacecraft. Muttniks is an open source sample dapp, built to guide you through our exciting journey with Astro Ledger. Like the real stars and planets featured on astroledger.org, you can securely adopt, name, and trade Ethereum space doggos with Muttniks (and then donate your testnet ETH to build more Laika monuments). The following is a step-by-step technical exploration of the inner workings of Astro Ledger, which you can recreate on your local machine with Muttniks.
Muttniks was the “pet” project of Astro Ledger CWM (Chief Wow Muttniks) Tony. Thanks Tony!
HD137052. It has been named ‘Tlon’.Keeping Track of Stars & Muttniks
In order to assign names to actual stars, we needed a way to uniquely identify them. For this we used the Henry Draper Catalog of the brightest stars in the sky, with meta-information provided by the HYG database from The Astronomy Nexus.
Our specially selected Muttniks have no such prior catalog so we pseudo-randomly assigned each Muttnik a number, 1–6. The meta-information about each Muttnik (name and image) is loaded into the database in this Flyway migrations file. Our Astro Ledger creative content (spearheaded by Chief Chocolate Dehydrator Aleeza) and image links are similarly loaded into the database, with SQL code created by offline Python scripts that load data from HYG and other sources.
When a Muttnik is added to a database, we use a Truffle migration script to add the same Muttnik to the contract, putting it up for adoption. For Astro Ledger stars, those same offline Python scripts create corresponding Truffle migration scripts to create stars and put them up for space-grant-supporting-auctions at the price of about a dollar.
Contract & Events: Creating and Adopting
The Muttniks are stored in a “pets” mapping where each pet has an address variable for the adopter, initialized to address(0), and a boolean variable exists to distinguish this pet from one that hasn’t been created yet.
The function createAdoptee is used by the Muttnik Kernel owners to create each Muttnik in the contract. This is typically done in batch with a Truffle migration script, but could be called by any function. Adoption happens via the adopt function, which checks to see that the Muttnik hasn’t already been adopted before assigning the Muttnik’s address to the adopter. Adoption events are emitted so the browser can inform the astronaut/cosmonaut of the adoption when the blockchain confirms it.
Contract & Events: Naming Muttniks
The obvious solution to handling Muttnik (and star) names was to store the name as a variable in the contract. To find the name, folks could call a view function (for zero gas) that would return the name. However, storing data in an Ethereum contract is never a cheap operation. Because Astro Ledger’s goal is to allow anyone to be able to assign meaningful names to stars, we implemented the assignName function using events instead. Names are forever stored on the blockchain as 32 bytes of data on a log of the event, but because they aren’t accessible to the Ethereum Virtual Machine (EVM), the gas cost to assign a name is almost negligible. This philosophically-motivated decision allowed us the exhilarating discovery that much of the existing software for Ethereum does not fully support event-logs, but we went to the edges of the universe to achieve a solution for pulling up the names quickly, as described later.
Web Stack
While Chief Rubber Taster Jonathan worked on the contract and the star database, Tony began building the Astro Ledger web stack using Play! for a web framework (ol’ reliable), and Postgres for the database (ol’ good enough for everything). At first, Tony chose Twirl to generate HTML pages of available stars. He quickly learned that the majority of the anticipated Astro Ledger user base had settled on a browser plugin called MetaMask to talk to Ethereum blockchains, and so began sneaking in some JavaScript (including a popular library called web3js) into the webpages to support this. The code to generate web pages soon became a funny mix of Scala, HTML, and ES5, where all of the blockchain interactions were executed using web3js. This worked, and was fast enough in our test environment.
Though it’s probably still fair to call the main Ethereum network itself a “test environment,” deploying our app to it in our first production alpha revealed one that was far slower than even Tony’s 2014 Macbook Air. This was particularly true for the for-loops we used in our contract to search for a user’s star cards, which were executed individually in the user’s web browser whenever they went to our page (note that, to guard against this performance hit, the Muttniks Solidity code uses no for-loops). To prevent expensive, redundant contract calls, we decided to move some of these queries to the Scala side of things, where they’d execute once on our Heroku instance every few minutes, and then be accessible to a user’s web browser without it having to call any contract code itself.
This was great for querying a user’s star cards, but things became interesting when we needed to query the names that they assigned to stars. As mentioned earlier, rather than put these names in contract storage as we did with auctions and card ownership, we chose to keep a record of star names using Ethereum events: this was intended as a less expensive form of storage. So why not cache these on our server? As we soon discovered, web3j (a library similar to web3js that we used for querying the blockchain in Scala) did not support filters on Infura, the free client we used for accessing the main Ethereum network. Web3js did support querying events in the user’s browser, but this had even worse performance effects on the browser than querying ownership, because it would poll the blockchain every second or so to get event information (web3js 1.0 has since addressed this).
To get a better handle on things, we decided to go closer to the source. In earlier investigations, Tony had coded up some Clojure code to talk directly to Infura’s RPC endpoints. With it running on our server, we could essentially do the same thing as web3js would in someone’s browser, but in one place (for $7/month) and at a longer poll interval. So, we decided to turn it into a full blown service. Coupled with Redis, this gave us our own centralized names cache, which, as with auctions and card ownership, could be accessed in someone’s browser much more quickly than if it were to query the blockchain directly itself.
Single-Page
In the meantime, Chief Rubber Taster Jonathan was tasting the rubber over in Tel Aviv when he met the team developing Portis, a browser based smart-contract enabled wallet. Portis would allow folks to use Ethereum on a vanilla web browser (i.e. without MetaMask installed), including iOS or Android. However, since the Portis library only stored user credentials in browser memory, our initial approach of rendering web pages server-side (where clicking a link would open a brand new page) would oblige users to log in over and over again whenever they navigated anywhere on our site. The solution was to migrate to a single page application that could be downloaded all at once in a user’s web browser, to provide full functionality without a user ever having to refresh the page or log in again.
To accomplish this, we settled on React, since it afforded us the ability to easily use another popular library, Drizzle-React to interoperate with web3js in a singly-paged way. However, The Drizzle library itself relied on web3js 1.0, which at the time was not fully compatible with MetaMask. We eventually abandoned Drizzle, but continued on with React.
Muttniks!
Muttniks is a pared down version of the Astro Ledger app, available here:
https://github.com/tlazenka/muttniks
To get started with Muttniks, head over to the cheat sheet! The overall structure can be gleaned from the docker compose file, and by drilling down into each service’s individual list of dependencies (note that some dependencies aren’t relevant to Muttniks, but were used by Astro Ledger). Docker compose is a convenient tool that lets you package up all these fun little services so you can easily launch them together.
Docker Compose Services
app
The main Scala app that sits atop Postgres. When the app is started, it runs the database migrations if needed with Flyway, and then Slick generates the case classes from the resulting schema. A periodic task is scheduled with Akka to query the blockchain with web3j and update the Postgres database.
db/redis
The Postgres/Redis databases.
jobs
The Clojure service that sits atop Redis. When started, it schedules a task with Quartzite to periodically query the local Ethereum blockchain, and caches the results in Redis using Carmine.
sbt/lein
Centos images that map sbt/ivy2/m2 directories to your local machine to speed up dependency downloads.
truffle
A Truffle service for compiling the Ethereum contract and running the tests.
ganache
A Ganache service used here for a local development blockchain.
react
The React version of the website.
web3j
A web3j image that includes the command line tools for generating code from the Truffle JSON artifacts.
firefox
A service with a Firefox browser plus MetaMask that you can access using VNC, you don’t want to install MetaMask in your own browser.
Wrap-Up
In case you are struggling for a reason why Muttniks needs to be on any blockchain… you are not alone in the universe. However, let’s suspend disbelief and say that the gray area of “ownership over the naming privileges of non-existing Soviet dogs long gone to the happy hunting ground in the sky” needs to be codified in some way…
You can put literally anything countable on blockchain!
We are really happy with how the framework runs and would love to hear about how Muttniks has helped you to put vegetating llamas on blockchain, or maybe adopt every single Amur Leopard on blockchain, or perhaps slam slowly slaloming snails… yes, on blockchain!!! If you are having any difficulty with your implementations, just radio for help. The Astro Ledger team would be happy to work with you — shoot an email to groundcontrol@astroledger.org.
And if you just can’t get enough Muttniks, don’t forget to beam over to astroledger.org and try naming some blockchain stars!
