A Crash Course in Lightning App Development

This guide is for anyone who wants to jump into programming on the Lightning Network without necessarily caring about doing things the “right,” “correct,” or “best” way. In other words, this is a highly-opinionated guide. If you’re looking for an O’Reilly-style handbook this is not it. On the other hand, if you want a summary flyover of a minimal Lightning development environment and its most essential parts, welcome aboard. My goal here is to distill the creation of a Lightning application down to its bare essentials so newcomers can get to building and experimenting quickly.

My belief is once you understand the basics, you can move on to writing more complex and secure applications. But everyone’s gotta start somewhere, and as I personally found the process of starting to be incredibly difficult, I thought I could save a few lost souls some pain by putting these thoughts together.

9/16/21 Edit: I published a follow-up to this post now with a detailed, technical implementation of a media store that builds on the concepts outlined in this post.

Before we get to the nitty gritty, a little background:

About a month ago, I left a steady job as a data architect to study and work on the rapidly growing Bitcoin ecosystem. If you’re here, maybe you’ve made (or are considering making) a similar leap.

Quite simply, I think Bitcoin is most fascinating technical innovation I have ever seen. And yes, that includes the internet. The more I learn about Bitcoin, the more enamored I get with it. This technology has the potential to transform and disrupt so much about how we transact today — for the better — and I want to be a part of that change.

That is why I chose to build on Bitcoin. My hope is this post will help educate developers who are not familiar with the Lightning ecosystem and its possibilities, and encourage more creative engineering in the space.

Honestly, there is so much about the Bitcoin ecosystem that I’d love to write about eventually (the concept of energy-backed money being at the top of that list), but we’ll save those discussions for another day. Right now, it’s time to start building.

Literally the worst font I could find

The Architecture of Lightning

via lnrouter.app

The basic design of the Lightning Network is a bunch of nodes connected to one another via channels. There are set amounts of bitcoin committed to those channels, which we record as a transaction on the main Bitcoin blockchain. Then, using special, off-chain contracts, we’re able to move those satoshis around between nodes without having to commit a new transaction to the main chain.

We do this by keeping track of the balances in channels as transactions take place. For example, if you have 100 sats committed to a channel with Alice’s node, you have about 100 sats (minus fees) you can spend with either Alice directly or anyone else you are able to connect with through Alice. These payments are near-instantaneous because all we have to do is adjust the balance of the channel with every transaction. This happens with the help of encrypted messages that travel across the network between sender and receiver.

Lightning applications operate against this so-called second layer of Bitcoin, the Lightning Network. This enables a developer to build an application that can operate using bitcoin as a payment method but with rapid settlement and lower fees.

Now, this is a vast oversimplification of the Lightning protocol, but I think it’s enough to start. If you want to understand in more detail how LN works, here are a few good sources. But be mindful that the rabbit hole goes deep, and if you’re not careful you will be reading about the ChaCha stream cipher family at 1 am with a code editor sitting empty alongside your browser. Not that that’s necessarily a bad thing, but we do want to build something eventually. And also sleep.

An even better way to learn about LN and how nodes work, in my opinion, is to set up a node yourself and figure out how to route payments through it. This takes a little more time and money but it can be fun as well as educational.¹

The Implementations

Lightning is an open protocol, which ultimately means there is a standard way to communicate and interpret messages over the Lightning network. In the same way that no one owns HTTPS or TCP/IP, no one owns the LN protocol and anyone can participate over LN as long as they follow the standard. Not only that, the standard continues to evolve as developers try to harden the protocol and add new features.

There are four companies at the heart of developing the protocol today: Lightning Labs, Blockstream, ACINQ, and Square Crypto. Each has their own implementation of Lightning:

Lightning Labs — lnd (Go)

Blockstream — c-lighting (C)

ACINQ — eclair (Scala)

Square Crypto — Rust Lightning (Rust)

All of these implementations are able to communicate with one another over LN. The differences lie in their individual API’s. This guide will focus solely on Lightning Labs’ lnd because that is what I started with and know best. It was a bit accidental in terms of how I ended up using lnd, but to their credit, the team’s documentation is pretty good and their Slack channel for developers has been great for getting help and support.

So be aware the rest of this guide will be lnd-specific, but the general concepts should apply to any Lightning implementation.

Your Development Environment

Two words: Use Polar.

Ok, a few more words. The biggest hurdle for anyone doing anything on Lightning right now, whether you’re a developer or a user, is getting your backend set up. Now I’m not talking about just a wallet, for which there are plenty of fast and easy custodial solutions. I mean a true backend — a node running Bitcoin and Lightning.

I hope the big mouse is OK with this.

The reason for this is your application needs to have certain permissions on a node in order to do important things like create invoices and watch for payments via API/gRPC calls. While this is possible if someone gives you access to their node, it would be good to have your own.

In order for a node to be functional for the purposes of Lightning development, it has to have 1) a running Bitcoin process with an up-to-date copy of the blockchain and 2) a running Lightning process with some open channels to send and receive payments through.

Over an average, residential internet connection, the blockchain sync alone can take several days or more. There is a lite, abridged version of Bitcoin you can run called Neutrino that can work, but I’ve also heard it can cause problems when trying to develop on Lightning. I’ve never worked with it personally. Either way, for someone just learning these tools I figure it’s best to remove variables that could cause unpredictable side effects, so just stick with the full blockchain if you can afford to.

Once we have the Bitcoin layer sorted, then there’s the Lightning layer. As I mentioned, we need channels in order to transact over LN. While it isn’t too difficult to open channels on Lightning, it not only requires some planning and coordination, but also requires bitcoin. You could, of course, commit some Bitcoin to open some channels and start playing around with transactions. But there’s no reason to take on such financial risk.²

Forget all that for now. Polar is an amazing tool that allows you to run a simulated version of all these processes in Docker containers on your laptop. It’s even got a slick UI that visualizes the network for you.

Look, Ma! No waiting for sync! (Polar screencap)

What’s great about Polar is that all you really need to do to switch your application to a testnet or mainnet backend when you’re ready is update a few configuration settings. I’ve been able to flip between local, testnet, and mainnet backends easily by just commenting out/in a few lines of code in my application (I will get to that .env file eventually. We’re not here to judge now, remember?).

Polar also supports lnd, c-lighting, and eclair nodes, which means you have the flexibility of trying other implementations if you like.

So to reiterate: Just use Polar. It’s a fast and easy way to get started, and who knows? Maybe you won’t even like developing on Lightning. Might be good to find that out before spending a lot of time and money on a system you’ll never use. On the other hand, if you find yourself listening to Stephan Livera podcasts nightly as you drift off to dream about preimages, you can set up a more serious dev environment.

Setting Up Polar

We can create our simulated Lightning environment in Polar quite easily by using the “Create Network” action. Name your network whatever you want (“test” is pretty clever, I think) and add 2 lnd nodes and 1 Bitcoin Core node. That’s really the minimum you need to start, and you can always add more later. Once you’ve created the network, start it up. The first time you run through this may take some time as you’ll have to download all the Docker images. Oh yeah, by the way, you’ll need Docker on your machine. Like I said before, we’re flying by the seat of our pants here.

After all that you should have a screen in Polar that resembles this:

In the beginning…

As you can see we’ve got Alice and Bob floating in space with their lnd nodes and both are hooked up to a Bitcoin Core backend process. Also note we’re at a block height of 1. Our very own Genesis block! Unlike a live blockchain, Polar only mines blocks when we need to commit transactions as we develop and test.

The next step is to create a channel between Alice and Bob so we can make Lightning payments. If we click on Alice’s node, the control panel on the right changes to show the different ways we can interact with this node. Under the “Actions” tab we’ll find all the tools we need to fund Alice with some simulated bitcoin and subsequently open a channel between her and Bob. So let’s deposit some funds to Alice and once that’s done, open a channel with Bob.

Click “Deposit” and proceed with putting 1M sats in Alice’s wallet.

If everything went according to plan, you should see the block height change as well as Alice’s balance. If not, seek help. It’s not your fault.³

Now we can open a channel between Alice and Bob. Since Alice has all the funds right now, we can simply open an “Outgoing” channel with Bob. This basically means Alice is committing a certain amount of bitcoin to spend with Bob through this channel, and they can use this channel for multiple transactions as long as there is a balance remaining. Note, however, that immediately after the channel opens, only Alice has the ability to pay Bob because the spendable balance is all on her side. That can change, of course, after Alice pays Bob.

This management of money in channels in order to maintain the ability for people to transact is what we mean when we talk about liquidity in Lightning. Practically speaking, users can only spend what is committed in all the channels across the network. Taking that a step further, a single payment can settle only if there is enough money in all the channels between the payer and the receiver (money going in the right direction, additionally). Otherwise that payment will fail. The management of and workarounds for the constraint of finite liquidity in LN is a topic in and of itself.

So opening channels in Polar… this is, admittedly, the place where I run into issues sometimes. I suspect it’s related to a state discrepancy between the UI and the backend, but I’m not sure. Stopping and restarting the nodes (or the entire network) sometimes helps. Quitting Polar entirely and restarting can do the trick, too. Also, in times like these, it’s amazing what a 10-minute walk outdoors can do for the mind and spirit.

Regardless of the reason, I’ve found the best way to run these node operations is via the CLI that Polar provides. While the UI might not always reflect the changes, we can be certain the database (so to speak⁴) is accurate by querying it directly. Polar makes this easy, so let’s do that for now.

Making sure we have Alice’s lnd node selected, let’s go to the “Actions” tab and click on “Launch” under “Terminal”. This should open a command prompt like so:

ASCII art circa 2020

Now we can use lnd’s lncli tool to open channels, create invoices, and pay them. First, let’s try running the following to get a lay of the land:

lncli --help

To open an outbound 100,000 sat channel between Alice and Bob, we can use the following:

lncli openchannel --node_key <bob's public node key> --local_amt 100000

You should see a response with a “funding_txid”. This corresponds to the transaction Alice and Bob are broadcasting to our simulated Bitcoin blockchain to commit the funds to our channel.

Try running the following to see our new channel:

lncli listchannels

If the list the command returns is empty, try mining a few blocks in the Bitcoin node (it’s under the “Actions” tab). This should help confirm the transaction and open the channel.

We can now see via the listchannels response that we have a channel with Bob and it should have a local balance of close to 100,000 sats (minus transaction fees). This local balance is what Alice can now use to pay Bob over Lightning.

The Payment Flow

As of today, the simplest (and I believe most common) payment flow over Lightning is via invoices. An invoice is essentially a set of payment instructions with two key components: how much and to whom. There are other parameters and variations on this theme that allow for some interesting possibilities (like hodl invoices and the BOLT12 proposal) but we will focus on this most basic of patterns here.

Continuing with our Polar setup, we start by creating an invoice. But this time, we’re going to use Bob’s node, so launch a Terminal for Bob and try running the following:

lncli addinvoice --amt 100

The command above creates an invoice for 100 sats (I actually ran into a connection error at this point while running through the steps. Try stopping and restarting Bob’s node if you have a similar issue). We can see via the response a bunch of info about this invoice:

{
    "r_hash": "7d91cafaba85b6086924142dfd890f350eb53b17b80e2993d0a2ce5ccc7252f1",
    "payment_request": "lnbcrt1u1ps3lu04pp50kgu4746skmqs6fyzsklmzg0x58t2wchhq8zny7s5t89enrj2tcsdqqcqzpgsp55rtlzlf5rt0z5zg34nc2rlcm9mw6nd77x45r85z6zp07qumphr7q9qyyssqzrvxdlsluaeu7esscvv8skcmaly4794j7pg9ytapmn50uukezf4xpqma9758s39wpn4pwk475dztezg4tff8xpylksl4mww57q8hj7cq7s7222",
    "add_index": "1",
    "payment_addr": "a0d7f17d341ade2a0911acf0a1ff1b2edda9b7de356833d05a105fe07361b8fc"
}

We’re only going to focus on the “payment_request” portion for now, because that piece of data contains everything we need for Alice to pay Bob — namely, the payment amount and where the money is going.⁵

If we flip back to Alice’s node terminal, we can take the payment request and pass it in as an argument to the following command:

lncli sendpayment --pay_req <payment_request>

And the result is:

Success.

Choose Your Own Adventure

I used to make diagrams like this for work. I still do, but I used to, too.

At this point, you should have the basic knowledge and tools to start building something. The diagram above outlines a simple example application that uses the same API calls we used to create and pay an invoice in Polar. It’s really all you need for a bare bones app. Of course, this is just one example, and there are lots of other concerns we can pile on as we develop — these are only starter blocks — but you can figure out what you need and how to make it along the way.

Some additional tips and references:

• There are a lot of libraries out there meant to help developers write less boilerplate code and cut right to the action. For me, learning how to use these libraries created more frustration than efficiency. That’s more on me than the libraries. Abstractions are great if you have a basic understanding of what is being abstracted (now that’s a t-shirt!), and I did not have that understanding when I started. Personally, I found the API docs for lnd the easiest to follow. Once I used this guide to configure a gRPC client for Javascript, I was off to the races.
• To see an example of a more fleshed out application, check out this excellent tutorial on Lightning Labs’ Builders Guide. I would recommend this tutorial if you’re familiar with the tools they use in it: express, mobx, and React. If you’re not familiar with these tools, you may not get as much out of the tutorial but you will still learn something. One thing I really like about it is it showcases a few interesting features that Lightning (and cryptographic proofs in general) can enable in an application.

Finally, please send me any feedback or questions about this post that might help improve its clarity and/or accuracy. To a point, of course (see first sentence).

Thanks for reading.

Let’s go.

¹ Umbrel is a good place to start if you want to run a node. I’ve also heard good things about MyNode, RaspiBolt, and RaspiBlitz. If you’re into tinkering with software systems (or SimCity), running a node is at worst a fun diversion and at best an all-consuming and never-ending game of optimization with actual financial stakes.

² There also is a testnet you can work on, with bitcoin faucets to get you started with some practice bitcoin. These coins are worthless, but it’s fun to pretend to be a digital Scrooge McDuck. One convenient way to stand-up a node if you’re not interested in configuring and managing it yourself is via a service called Voltage. It operates using a cloud-based, pay-as-you-go model and supports both testnet and mainnet nodes.

³ Well, it might be — we don’t know yet. In all seriousness, check out the lnd Slack for developers. I have run across many of the devs who build and maintain these tools there. You might even run into me! Chances are someone has seen the problem you’re facing. And if not, great — more to learn for everyone.

⁴ The concept of blockchains as a public database where everyone has root access is an intriguing one. More on that: https://balajis.com/yes-you-may-need-a-blockchain/

⁵ For a deeper explanation on the other fields in an invoice, there is a good summary here. For more on the underlying contract at play, read up on HTLC’s.