A developer experience journey
TL;DR We are developing a framework that allows you to write Bitcoin applications as if you were writing web applications. You can find a full-featured example here.
Since the release of Bitcoin, developers and researchers have been looking into different ways to use the blockchain. Companies have been interested in using blockchain technology for a wide variety of applications, from issuing assets to tracking products through their supply chain, the use cases seem to be endless. A lot of these companies have started developing their own blockchains, like Ethereum and Hyperledger Fabric, in order to solve Bitcoin’s perceived problems.
This evolution has raised numerous questions. Are these blockchains well designed? Are these blockchains able to scale? Aren’t we reinventing data silos by using application-specific private blockchain platforms? The world has been told Bitcoin isn’t meant to be used as a blockchain backbone for application development, but is this true? Among other things, we’re told writing software for Bitcoin is hard and impractical. Let’s find out.
“Bitcoin development doesn’t have to be difficult”
This article is a developers’ short story on how we attempt to make Bitcoin development as easy as developing a simple web application.
We looked into some of the design patterns developers use to create traditional web applications and tried to apply these to Bitcoin.
Developer experience
With over 9 million downloads per week, Express.js is one of the most popular NPM packages today. Express.js is a framework for writing web applications and APIs. It abstracts away most of the complexities of HTTP servers and allows developers to focus on their application.
Looking at a code snippet, it’s easy to see why this framework is so popular.
While this is a convoluted example, it shows a few things:
1) It’s easy to attach behaviour to a specific request (or route)
2) With middleware we can inject reusable behaviour, keeping the final function handlers clean and compact.
This design pattern is known as the chain-of-responsibility pattern. It decouples the sender (HTTP server) from the receiver (the handler functions) and allows for more than one receiver to process a request.
In this case, the request is an HTTP request, but it doesn’t have to be.
A universal middleware router
To create the same developer experience on Bitcoin, we want to handle Bitcoin transactions in the same way we handle HTTP requests. To do this, we started to rip out all the HTTP-specific parts from Express.js. However, we soon realized that Express is quite big and very tightly intertwined with HTTP. The much simpler Connect.js provides the same basic middleware functionality as Express.js, so we decided on using Connect.js as a base for a transport-agnostic middleware router.
We end up with the following API:
The HTTP-specific `listen()` function has been removed because this middleware router is transport-agnostic. We will later implement this function in a higher-order module that listens to Bitcoin transactions.
Listening to Bitcoin
With the universal middleware router in place, all we need to do now is listen for Bitcoin transactions and call `handle()` with the transaction as the first argument. Essentially, we are reimplementing the `listen()` function.
The idea is to replay all existing transactions to our application and then listen to all transactions in realtime. A simple state machine application (see example) will then reach the same state every time it syncs with the blockchain.
Though you could listen to each and every transaction in theory, in practice most of the time you only want to listen to the subset of transactions that is relevant to your application.
In Express or Connect, `listen()`’s first argument is usually a TCP port number. This essentially acts as a filter: the application is bound to the TCP port and will only receive traffic destined to that port. For our purposes, however, `listen()` should accept some sort of filter as an argument. The filter expresses which transactions we are interested in.
Bitcom, BitDB and BitQuery
An interesting de facto standard has emerged recently, called Bitcom. Bitcom is a `Decentralized Global Registry of Bitcoin Application Protocols`. The idea is that you prefix all OP_RETURN data (OP_RETURN is the name of a technique used to publish data through Bitcoin transactions) of a certain protocol or application with a unique Bitcoin address. This makes it easy to identify and filter protocols. You can read more about Bitcom here.
We use BitDB to query existing transactions and BitSocket to listen for realtime transactions. Calling the `listen()` with a Bitcom prefix like this will make the application listen to OP_RETURN transactions beginning with this prefix.
For those familiar with BitQuery, it’s good to know that `listen()` still accepts raw BitQueries as well. There are still some technical kinks we want to iron out here, such as abstracting away the difference between a BitSocket and a BitDB query. Keep in mind this is still alpha software
A quick detour: Bitcoin transactions as plain objects
In raw form, Bitcoin transactions are a long string of zeroes and ones. Usually, they are represented as a hexadecimal string.
In this form, Bitcoin transactions are unwieldy. We want to interact with an incoming transaction as easily as we interact with HTTP requests in Express. Therefore this hexadecimal blob needs to be decoded into something friendlier.
This is a lot better. But the juicy parts are in the `asm` fields of `scriptSig` and `scriptPubKey`, and they are still just strings.
TXO to the rescue
Thankfully there exists a better way of representing Bitcoin transactions: TXO (transaction object). In a TXO, every operation in the Bitcoin script can be accessed individually and in multiple formats. See here if you want to learn more about TXO.
Here is the list of outputs of the previous transaction in TXO format:
Routing without routes
Normally, an HTTP resource (such as ‘/hello’) would be associated with a request. Inside an Express application, we call this a route. When an HTTP request is received by the server, the route is derived from the requested HTTP resource and the (application level) request gets sent down the chain-of-responsibility.
But since the plan is to push Bitcoin transactions through the handler chain, who or what decides the route? Short answer: You do. Thanks to the magic of middleware, you can modify `req.route` to be anything you want. `req.route` is the magic variable that decides which handler processes the request.
For example, let’s say we want to handle two kinds of transactions: `increment` and `decrement`. In the OP_RETURN of the transaction, we expect `s2` to be either `increment` or `decrement`.
Simple RPC
There are many ways you can define messages that you can parse as RPC commands. The most basic example would be an OP_RETURN that looks like this:
Alternatively, you could do this:
where the encapsulated message could be any protocol of your choosing such as JSON (`[ fn, arg1, arg2, arg3, … ]`) or Protocol Buffers.
Both have pros and cons. The first method allows for easy querying with BitDB for example. The second method, where everything is inside the first OP_RETURN argument, lends itself more to being encapsulated by other encoding or encryption layers.
The only thing needed to set this up in an application would be a middleware function that parses the messages and fills `req.route` with `fn` and `req.args` with all the arguments. In addition `req.caller` can be filled with the address of the transaction’s sender.
Because we see this as common functionality, we added standard middleware to our library. This way it’s easy to get started with Bitcoin development.
Here’s an example of an OP_RETURN router:
Middleware as authentication layer or paywall
The possible features of middleware aren’t limited to merely routing requests or building an RPC system. In fact, all of the transactions’ data can be used to build specific middleware functions.
In this example, the paymentRequired middleware checks if a payment of minimum 500 satoshis was made to a specific Bitcoin address, before executing the content of the handler.
ECIES decryption
Using the ECIES encryption scheme we can encrypt messages to the contract agent. Using middleware, those messages will be decrypted and the rest of the code can be implemented as usual.
In this example, we use the decrypt middleware to automatically decrypt all requests, before continuing execution of the handlers.
Conclusion
Bitcoin development doesn’t have to be difficult. By abstracting away transaction handling and raw data formats while adding the chain-of-responsibility design pattern, developers can build applications the way they were always used to. Luckily you don’t have to build any of this yourself, as we’ve released Casco as an open-source framework.
You’re used to building websites, REST or other APIs?
Great, you can start building on Bitcoin… today!
Visit our GitHub for more details about Casco!
