Part 18: cyphernode I

Preamble

Up to this point, we have looked to handle everything ourself, you know with “self-sovereignty” and all. That said over the course of the year (or so) we have run into a number of promising projects that were aligned with our goals whilst simultaneously making the software stack for Bitcoin easier to install, use and maintain. This is music to ears so we thought let’s play with a few of them and see what we leverage the output of others much smarter than ourselves.

Introduction

The first one we are going to look at is “cyphernode” is a modular Bitcoin full-node microservices API server architecture and utility toolkit to build scalable, secure and featureful apps and services without trusted third parties. It is a docker container built and maintained by “Bullbitcoin”

The code can be found “here” in the Satoshi Portal repository.

*Note they used to have satoshiportal.com and I assume that is why this project is under the Satoshi Portal GitHub but the URL now points to “bullbitcoin.com” A tad confusing I know but it is a young project and I assume they are in the process of figuring out the branding and such.

It is worth throwing some semantics out there. Currently, we have a backend (admin etc), API, frontend etc. Here we are basically splitting the backend in two and adding Bitcoin Backend and ECS backend (unless we think of a cooler name).

Under the hood

As we said above “cyphernode” is a collection of software put inside a docker container for easy maintenance and deployment. So let’s go ahead and see what each of the components that make up “cyphernode” does. the following image (taking from their “Github”) shows how each of the components works together

• Note We have failed to get the API working. The “docs” show connections to http://cyphernode:8888/ which we have tried this as well as http://127.0.0.1/ and with the port 2009 (as shown in the diagram above). Maybe we are not supposed to use the API and it is just for their stack or maybe we are just dumb and have not figured it out yet (most likely)

Below is a breakdown of each of the components that make up the stack.

traefik

A reverse proxy/load balancer that’s easy, dynamic, automatic, fast, full-featured, open-source, production-proven, provides metrics and integrates with every major cluster technology

gatekeeper

front door where all requests hit Cyphernode. Takes care of: TLS, authentication and authorization.

proxy

Request handler. Will dispatch authenticated and authorized requests to the right component. Uses an SQLite3 database for its tasks.

proxycron

Scheduler. Can call the proxy on regular interval for asynchronous tasks like payment notifications on watches, callbacks when OTS files are ready, etc.

broker

pub/sub mechanism is taken care of by the broker to which all subscribers and publishers should register.

notifier

Handling callbacks used by watchers as well as OTS stamping.

pycoin

Bitcoin keys and addresses tool. Used by Cyphernode to derive addresses from an xPub and a derivation path.otsclient

“Open timestamp client”
Used to stamp hashes on the Bitcoin blockchain.

bitcoin

“Bitcoin Core” version 0.18.0 (as of writing)

lightning

“C-Lightning” to version 0.7.1

sparkwallet

“sparkwallet” is a minimalistic wallet GUI for c-lightning, accessible over the web or through mobile and desktop apps

As we continue to use “cyphernode” we will become more familiar with the above components and loop back and update the descriptors accordingly.

Installing cyphernode

First, let us install it locally to make sure it all works as expected.

Step 1: Create a directory

$ mkdir cyphernode && cd cyphernode

Step 2: Run quick install of cyphernode

$ curl -fsSL https://raw.githubusercontent.com/SatoshiPortal/cyphernode/master/dist/setup.sh -o setup_cyphernode.sh && chmod +x setup_cyphernode.sh && ./setup_cyphernode.sh

Step 3: run through cyphernode set up

The first thing you will see a progress window (as shown in the screenshot below) that is basically the cyphernode set up downloading everything it requires to complete the installation.

Next, you will see is a prompt asking you for a password add one, obviously. Remember this password as it comes in handy (a lot) later.

After that you are presented with is the one below, press return as help is always good, right?

Next, we want to install both Lightning and Opentimestamps client so press space on both of the options and then press return.

Next, we want to select Testnet as we are testing.

Next, It asks to change user which is very good advice which we are going to ignore for the sake of simplicity. Go ahead and select N

Next, select “N”. I will be honest with you we are still not 100% sure what is “xPub” and until we do and can make an informed decision about it we are going to ignore its existence.

Next, We have to give an encryption passphrase that gatekeeper will use for authentication. Go ahead and add one.

Next, select a port. 2009 is fine so just press return

Next, set up the IP’s we are going to run everything over 127.0.0.1 for now so just press return.

Again just go for the default 127.0.0.1

Give it a name

Give it a colour

Now, It is telling us it will look after the full node for us yay. Press return.

Next, we have to add an RPC username. “admin” is always good

Next, we have to add an RPC password.

Next, type “N’ as we want a full node that is not in “prune” mode so we can fully validate transactions.

Next, you can add a UA comment here if you want (we did not)

Next, Press return as we are presented with one option and luckily it is the option we want.

Next, you are going to be asked for the location of where to put the files just press return 6 times as the defaults are good and you will be presented with the below

Next, We want to expose the gatekeeper so just press “Y” and then return

Next, again we want to expose the full node so press “Y” followed by return again.

We want Lignthing to be accessible outside the Docker network so select “Y” followed by return.

Next, you want to choose from docker-compose or docker swarm. We selected docker-compose and suggest you do the same.

Next, type “Y” as it is always good to clean up after yourself.

Finally, you will see some output (like the screenshot below) that is the installation completed.

Once it is finished you will see the following

Update cyphernode

To upgrade to the most recent version, just get and run the most recent version of the setup.sh file as described in the previous section. Migration should be taken care of by the script.

Your proxy’s database won’t be lost. Migration scripts are taking care of automatically migrating the database when starting the proxy.

Change cyphernode configuration

If you would like to make some changes such as add/remove lightning then you simply the type in the command below.

./setup_cyphernode.sh

This will ask you for your password and run you through the set up one more time quickly allowing you to make changes.

*Note, if you enable Lightning in a pruned node this will cause the errors we listed below as Lighting (in cyphernode at the moment) cannot be run on a pruned node so will require a rescan. Support for this was added to Bitcoin Core 0.18.0 so I am sure support will be added soon.

Using cyphernode

Starting cyphernode

The first thing we have to do is to start “cyphernode” to do this go to your terminal and type the following

./start.sh

and you will get a screenshot similar to the one below

*Note, sometimes you will get an error with “Testing Bitcoin” or “Testing Lightning”. This appears to be an issue with the respective blockchains syncing and if you stop it and start cyphernode again it eventually works.

The screenshots below show the most common errors you. will encounter whilst running “cyphernode”

This means that Bitcoin has failed to start.

This means that Notifier failed to start.

This means that Lightning has failed to start.

if you see an error like the one above type the command below and it will show you all the output and you should see an error related to Bitcoin.

docker-compose logs

In our experiecne, these happen when there is an issue with the blockchain such as it has to be rescanned or something and the simplest solution is to delete the testnet3 directory which can be found in the following directory.

/cyphernode/bitcoin/

If you delete this folder and run “./start.sh” again it should work. Although you will have to download the full blockchain again.

Stopping cyphernode

To stop the “cyphernode” simply type the following command in terminal

./stop.sh

Once it stops you will see the following out in the terminal.

The Admin

Yay, now we have it up and running let us log into the admin and check this its all working as excepted.

You may see the https issue (as shown below) if so just click advanced and then proceed to site.

The next thing you have to do is add your username and password that you selected during setup.

Next, you will see the screen below. There is not much we can do with the admin at present. I assume they have big plans for it in the future but the key thing is the progress bar as we want a fully synced blockchain before it can be of any real use to us.

Testing the RPC

Remember the RPC? We covered in in part 8 which you can review “here” well we should be able to call it here. So let us go ahead and try it. Type the following command into the terminal

curl --user admin  --data-binary '{"jsonrpc": "1.0", "id":"curltest", "method": "getblockchaininfo", "params": [] }'     -H 'content-type: text/plain;' http://127.0.0.1:18332/

You will be asked for your password and once you enter it you will get something like the following response.

{
   "result":{
      "chain":"test",
      "blocks":313988,
      "headers":1572503,
      "bestblockhash":"00000000000000e1eb33f4e1624f014693b233120fee8ad8e48becf7b1915dd1",
      "difficulty":3020938.8186549,
      "mediantime":1418445102,
      "verificationprogress":0.07199333815247766,
      "initialblockdownload":true,
      "chainwork":"00000000000000000000000000000000000000000000000139bde18ff233cec0",
      "size_on_disk":1608549685,
      "pruned":false,
      "softforks":[
         {
            "id":"bip34",
            "version":2,
            "reject":{
               "status":true
            }
         },
         {
            "id":"bip66",
            "version":3,
            "reject":{
               "status":false
            }
         },
         {
            "id":"bip65",
            "version":4,
            "reject":{
               "status":false
            }
         }
      ],
      "bip9_softforks":{
         "csv":{
            "status":"defined",
            "startTime":1456790400,
            "timeout":1493596800,
            "since":0
         },
         "segwit":{
            "status":"defined",
            "startTime":1462060800,
            "timeout":1493596800,
            "since":0
         }
      },
      "warnings":""
   },
   "error":null,
   "id":"curltest"
}

Testing the REST

REST is disabled by default but we can easily enable it as we did in the previous “tutorial” though we mostly use the RPC so it is not essential to do so.

Testing the API

*note as we mentioned above we were unable to get the API working but as soon as we do we will loop back and update this.

Conclusion

We have successfully set up and used “cyphernode” and it is impressive. It is still a very young piece of software and it still works extremely well. This is why we are going to use this as the core for “ECS” going forward and as it is using standard “RPC” calls this will require not much effort to fully integrate (isn’t Bitcoin brilliant?). This is what we will do in the next article.