jq oneliners for Lightning Network

I’m currently testing the c-lightning BOLT implementation and, being an incipient technology and all that, the tooling around it is not yet that much developed. This means that, for the moment, we have to heavily rely on the cli interface. And the best part is that all the command outputs are in JSON which is also a bit bad (because humans can’t easily read it, especially if it’s a long JSON), but it’s mostly good because we can use other tools at our disposal to make it more readable. One of these tools is jq.

First thing that jq can do is beautify the output, i.e. transform something like

{ "id" : "0360a66958608a187e7d0c3e9f26ad445c60b9b649fc2674c0d25fc83ce82d850b", "port" : 9735, "address" :
 [
  { "type" : "ipv4", "address" : "188.166.99.121", "port" : 9735 } ], "version" : "v0.5.2-2016-11-21-2037-g1192f16", "blockheight" : 1286394, "network" : "testnet" }

into

{
  "id": "0360a66958608a187e7d0c3e9f26ad445c60b9b649fc2674c0d25fc83ce82d850b",
  "port": 9735,
  "address": [
    {
      "type": "ipv4",
      "address": "188.166.99.121",
      "port": 9735
    }
  ],
  "version": "v0.5.2-2016-11-21-2037-g1192f16",
  "blockheight": 1286394,
  "network": "testnet"
}

But the most interesting features are those of parsing the JSON and selecting only parts of the document. For instance, using c-lightning’s cli getting a list of all peers one has to do

cli/lightning-cli listpeers

which would result in something like this (already beautified using jq by simply piping the output i.e. ‘cli/lightning-cli listpeers|jq’)

{
  "peers": [
    {
      "id": "03e103d5679dc41b96a0d11d8a9c359f9ee6f2eb3ba0e14e72d2b1918d03439d72",
      "connected": true,
      "netaddr": [
        "188.166.39.247:9735"
      ],
      "channels": [
        {
          "state": "CHANNELD_NORMAL",
          "owner": "lightning_channeld",
          "short_channel_id": "1284270:174:0",
          "funding_txid": "40cc700b1dd51f1eb5ed16f3fbc0de21807946b77ab0e4cca057b45bc379a281",
          "msatoshi_to_us": 1000000000,
          "msatoshi_total": 1000000000,
          "dust_limit_satoshis": 546,
          "max_htlc_value_in_flight_msat": 18446744073709552000,
          "channel_reserve_satoshis": 0,
          "htlc_minimum_msat": 0,
          "to_self_delay": 6,
          "max_accepted_htlcs": 483
        }
      ]
    },
    {
      "id": "033dd7f388fdbf33915c68b548883b899516a8ed7da14ae5eff03ad9f9ab1fe7ad",
      "connected": true,
      "netaddr": [
        "67.180.192.3:9735"
      ],
      "channels": [
        {
          "state": "CHANNELD_NORMAL",
          "owner": "lightning_channeld",
          "short_channel_id": "1284270:175:0",
          "funding_txid": "ec84b4163e6ccf0a0859fb57d441fcb9b99d4abb18dc28fd8d14413cb45a7351",
          "msatoshi_to_us": 991979118,
          "msatoshi_total": 1000000000,
          "dust_limit_satoshis": 546,
          "max_htlc_value_in_flight_msat": 18446744073709552000,
          "channel_reserve_satoshis": 0,
          "htlc_minimum_msat": 0,
          "to_self_delay": 6,
          "max_accepted_htlcs": 483
        }
      ]
    },
....
]
}

But this output, while very structured, makes it hard for a person to answer a question like “is peer XXXXXX connected or not”?

That’s why some transformation using jq can be useful. So for instance

cli/lightning-cli listpeers | jq -r '.peers | to_entries[] | select ((.value.channels|length)>0)| .value.channels[].state +" with peer "+  .value.id'

will list one line per peer with just the state, like this

CHANNELD_NORMAL with peer 03e103d5679dc41b96a0d11d8a9c359f9ee6f2eb3ba0e14e72d2b1918d03439d72
CHANNELD_NORMAL with peer 033dd7f388fdbf33915c68b548883b899516a8ed7da14ae5eff03ad9f9ab1fe7ad
CHANNELD_NORMAL with peer 027e92b4fa1988ae5b0d44b87da80ab8875312fdae6631a8896851bc41857e66aa
CHANNELD_NORMAL with peer 034fd1f523d694f340f2ee92971a04f837df3e910568222b889ad51b4b826ff361
CHANNELD_AWAITING_LOCKIN with peer 03c977ccfde5f30b43bcadd69dc1a1e8a29359824b8bddd44d9bf526a2b3f2bf37
CHANNELD_AWAITING_LOCKIN with peer 02f21e5594a92b26f1517906600bc5cef1fe6f8c39620ebb48a11c55661f3786ce
CHANNELD_AWAITING_LOCKIN with peer 03172377a939b21aadab64943ac82fc54a6600db48812dec9046c78abd78f5a0a2
CHANNELD_NORMAL with peer 0215b7a76fba355117f916473509719c5e9c3b57a0092f1395ceeea41f4b03c4f0
CHANNELD_NORMAL with peer 028fd42a3eb383bfb803e49f393fb11786108d879fcd46cdfa5ee283e3b896077b
ONCHAIND_OUR_UNILATERAL with peer 0340b8665eb88e576a8e6ec975ab1aa2aea1568deb7cd2195ccb83473f6e1fbf6d

Randomly connecting to peers

One of the problems when setting up a node and, more important, trying to pay from it, is that you need to have a few connections open (and channels funded) with some peers. If you don’t care who you connect to, you can use the following script to connect to 5 random peers and fund each of the opened channels with 100000 satoshi

cli/lightning-cli listnodes | jq --raw-output '.nodes | .[]  | select ((.addresses|length)>0) | select ((.addresses|.[0].type)=="ipv4") |  "timeout 5 cli/lightning-cli  connect " + .nodeid + " " + .addresses[0].address + ":" + (.addresses[0].port|tostring)'  | shuf -n 5 | bash  | jq --raw-output ' "cli/lightning-cli fundchannel " + .id + " 1000000"' |bash

(while for testnet this is ok, it’s probably not recommended to do this on the mainnet)

If you wonder what is the “timeout” doing in the command, the explanation is this — connect takes sometimes a looooong time, but most (if not all) connects that take more than 2–3 seconds eventually fail, so that’s why when doing connect we only wait 5 seconds and then we move on

The last utility script is for automatic close of channels in a specific state (in the example OPENINGD)

cli/lightning-cli  listpeers | jq -c '.peers | to_entries[] | ((.value.channels|length)>0)| select (.value.channels[].state=="OPENINGD") | .value.id' | xargs -n1 cli/lightning-cli close