Building the World’s First Open Source Multi-Sig Bitcoin Exchange

How you can have your own crypto-currency exchange with multi-sig security

Bitcoin’s history is rife with drama about the closure of one exchange after another. Problems have ranged from outsider hacking attacks to insiders holding servers hostage. I've long been curious to understand these stories better — How do exchanges work behind the scenes? How much would it cost to set one up? And if that were cheap, how would one run an exchange and stay in business (without losing customer funds)?

Like many others, I had long toyed with the idea of building my own exchange. A few months ago, I registered the domain 21million.club, because there would only ever be 21 million bitcoins, and buying one would make you part of the club. I never got around to building the exchange due to time constraints. Also, for practical and security reasons, I didn’t want to re-invent the wheel.

Just recently, I was pleasantly surprised when friends over at BitSpark (a multi-currency exchange in Hong Kong) told me about Peatio, an open-source crypto-currency exchange that was free for anyone to download and run. It boasts several enterprise exchange features, including a high-performance matching engine, scalable distributed worker threads, and SMS 2-factor authentication out of the box. It also has several features relevant to crypto-currencies, such as integrated withdrawals and deposits to Bitcoind and a built-in proof-of-solvency audit. However, one security feature missing was support for a multi-sig cosigner. By default, Peatio connects to a BitcoinD instance with an unlocked wallet — this meant that any hacker with access to that machine would be able to get away with all the coins.

Peatio is great as an exchange, but I feared for those who ran it in production — would security on their machines be strong enough to prevent hackers from repeating history in the long run? Since I work at BitGo, I decided to embark on a side project to enable multi-sig support in Peatio. I wanted to do this as a challenge to see how difficult it was to get enterprise-grade security on open-source exchange software. If it were possible, I’d be a step closer to running my own exchange☺.

The goal was to hook Peatio up such that withdrawals and deposits would continue as usual during normal operations. The exchange software would sign one half of the transaction, and BitGo the other half, but with the added safety of co-signing policies. For example, if the number of withdrawals exceeded a daily limit, or the withdrawal requests came from an unknown IP, the co-signer could require additional steps to sign the transaction.

The following steps detail my journey and provide instructions on how anyone can now have an opensource exchange with multi-sig support.

Step 1: Set up Peatio

Using a fresh Linode Ubuntu instance with 4GB ram and 96gb SSD, I began by installing Ruby, Mysql, Redis, RabbitMQ, and BitcoinD. Then, I followed the instructions here to install Peatio. I hooked up the Bitcoind RPC in the currencies.yml file:

- id: 2
 coin: true
 quick_withdraw_max: 1000
 key: satoshi
 code: btc
 rpc: http://rpcuser:rpcpassword@127.0.0.1:18332

Deposits page in my new exchange, the 21 million club. The address starts with m, which means it is controlled by a single key.

This worked! I was able to sign up for a customer account on the exchange and get a deposit address from the local bitcoind (note the address begins with m, a single-key address on testnet).

Step 2: Install BitGo Multi-Sig

Recently, during the integration with Bitstamp, BitGo built a drop-in replacement for BitcoinD. Called BitGoD, it responds to the same RPC commands as BitcoinD, including getbalance, listtransactions, listunspent and sendtoaddress. Since Peatio also uses BitcoinD in the backend, I saw this as the simplest integration path.

I installed BitGoD by cloning the open source project from git:

git clone https://github.com/BitGo/bitgod.git
cd bitgod
npm install

I inspected the Peatio code and identified that it was calling bitcoind expecting “payment” as the account name. BitGoD needed to be able to handle this:

bin/bitgod -masqueradeaccount=payment

Finally, I made a change to Peatio’s currency.yml file, pointing the RPC port to 19332 (which BitGoD was listening on)

- id: 2
 coin: true
 quick_withdraw_max: 1000
 key: satoshi
 code: btc
 rpc: http://rpcuser:rpcpassword@127.0.0.1:19332

I had to restart the Peatio daemons after this.

Step 3: Connecting the Multi-Sig Wallet

I excitedly went to the deposits page in Peatio, hoping to see a multi-sig deposit address created. This failed as BitGoD was running, but not yet connected to a wallet on BitGo (doh!).

To connect BitGoD to BitGo, an access token and a BitGo multisig wallet is required. To get the access token, create a Bitgo test account for the integration and head down to the Settings — >Developers tab. Then you can connect to the account using

bitcoin-cli -rpcport=19332 settoken [YourToken]

Next, create a BitGo wallet (from within the BitGo interface) and tell BitGoD the wallet address:

bitcoin-cli -rpcport=19332 setwallet [YourWalletAddress]

Deposits page showing multi-sig addresses generated (starting with 2)

I had to create a new customer account on the exchange to generate a new address. From then on, the exchange gave out multi-sig addresses for deposits (as you can see in the example, the address starts with 2)!

Step 4: Bitcoin Deposit Integration

Although the deposit addresses were generating and showing up fine, incoming Bitcoin deposits to the addresses were not reflecting on the customer’s balance. The reason for that was that Peatio was not being notified of the incoming transactions.

To inform Peatio about the new coin deposits when they occurred, I added app/controllers/webhooks_controller.js to accept an incoming webhook:

class WebhooksController < ApplicationController
before_action :auth_anybody!
 skip_before_filter :verify_authenticity_token
def tx
 if params[:type] == "transaction" && params[:hash].present?
   AMQPQueue.enqueue(:deposit_coin, txid: params[:hash], channel_key: "satoshi")
   render :json => { :status => "queued" }
 end
end

(Note how Peatio only accepts the Transaction ID— the deposit_coins daemon uses this to kick off checks on the deposit address and then verifies the transaction amount)

Here is the new route in config/routes.rb:

post '/webhooks/tx' => 'webhooks#tx'

After restarting Peatio, I then set up BitGo to send transaction notifications to the new route using the wallet Webhook API:

WALLETID=2NEe9QhKPB2gnQLB3hffMuDcoFKZFjHYJYx
ACCESS_TOKEN=9e1194fd035e2c8d5268e648c796425429fc2bd57bb5da7fbebbf09e1711a6b6
curl -X POST \
-H “Content-Type: application/json” \
-H “Authorization: Bearer $ACCESS_TOKEN” \
-d “{ \”url\”: \”http://www.21million.club/webhooks/tx\", \”type\”: \”transaction\” }” \
https://test.bitgo.com/api/v1/wallet/$WALLETID/webhooks

I sent some coins in, and they showed up on my customer account in the exchange!

The transaction deposit was successfully handled by Peatio’s deposit daemon and credited to the customer

I waited a while for the deposit to be confirmed. The exchange software polled the transaction until it saw a confirmation before updating the customer account balance. It even sent me an email! ☺

Step 5: Withdrawals and Policy Limits

Now that customer multi-sig deposits were being handled, it was time to integrate withdrawals from the wallet. In order to do this, I needed to give BitGoD the wallet passphrase (used when creating the BitGo multi-sig wallet):

bitcoin-cli -rpcport=19332 walletpassphrase [passphrase] 1500000

Some background: when sending a transaction, BitGoD decrypts the user encrypted key with the wallet passphrase, creates/signs the transaction and sends it to BitGo for co-signing and transmission to the Bitcoin network. Since key decryption and signing is done locally, BitGo never has access to the user’s private key.

Customer’s withdrawal page in the exchange

Withdrawals worked seamlessly in the exchange through the multi-sig wallet. I also set up a 2BTC policy limit on the wallet. In the above example, the 0.2BTC withdrawal was processed immediately, with the larger request pending approval by an additional user (such as the CTO or CEO) in the BitGo interface.

Approvals page in the BitGo interface

Such transactions require a human administrator to log in and input their OTP and wallet passphrase before being co-signed . This prevents theft by stealing private keys. I pondered developing a callback policy (through another route) to have BitGo perform an additional check with Peatio on whether the transaction matches the user balance... but decided to leave that optimization to a later time.

Final Thoughts

I had fun playing with Peatio as a free open-source wallet solution. As I was new to Ruby/Rails, it did take me some time to figure out the inner workings of the code and write the new webhook routes, but I felt it rewarding to be able to play with an exchange of my own.

I had a little fun with the trading engine too

We live in interesting times. Thanks to open-source, we now have free access to tried and tested exchange software to run businesses on. With Bitcoin and P2SH (multi-sig), it’s now possible to have multiple parties approve a transaction before funds get sent, while never putting control of funds in the hands of the security provider. There are few reasons left to code your own trading system or implement your own wallet security.

Build your exchange on a stable foundation and your customers will thank you for it in the long run. Let’s all stay a part of the 21 million club!