Inserting metadata into the blockchain.

Will Hill
6 min readJul 1, 2016

From a node/javascript developer’s view.

The aim is a terse explanation to fellow programmers and entrepreneurs, detailing how to programmatically move bitcoin from one address to another while in the same transaction injecting metadata. How immutable this metadata is and what it represents is down to you.

Take a look at how some companies are inserting metadata into the blockchain here or from the links below…

To read about the other facets of bitcoin and the blockchain that I’m not covering here checkout bitcoin.stackexchange.com.

The Value of a Bitcoin and Satoshi

To make a transaction some amount of bitcoin, however small, has to move from one address to another.

Just like a dollar is split into 100 cents, one bitcoin is split into 100000000 Satoshi’s. All transactions are detailed in Satoshi’s

577 USD* = 1 Bitcoin.
1 Satoshi = 0.00000001 Bitcoin. (10^-8)

1,000 Satoshi = 0.0057492000 USD.
0.00001000 Bitcoin = 1,000 Satoshi.

Check today’s Satoshi exchange rate here
or
Check the historical chart of the
bitcoin price here

*subject to today’s market

Bitcoin address

With email addresses, you send and receive emails and with bitcoin addresses you send and receive bitcoins. A bitcoin address looks like the below…

11WBguy6KVyTGnF4KX7Vmdx8ztj4wENh4W

The address can be shared publicly either just as shown above or converted into a QR code. The Address is also referred to as a public key.

To be able to make a transaction from an address you need to have the private key for that address which looks like the below. The format of the key is often referred to as WIF (wallet interchange format)

KwFmrKDWXT6oNK9QosHTU3V26EPT9ExVbq7n8Rs613ARsvr8vsW

The key must be kept a secret just like the password to your email address. The address and the private key are often referred to as a “key pair” you must keep the key safe, you can generate the address from the WIF anytime you need it but you cant generate the WIF from the address.

Here, a bitcoin address is more similar to a bank account. When you first create a new account the balance would be zero. You then need to deposit money into it in order to start sending money.

The only difference is that your bank knows who you are. Your bitcoin address is not publicly linked to your personal identity.
Unlike banks there is no “Know Your Customer” (KYC) or “Enhanced due diligence” (EDD) for bitcoin.

Deep dive on address’s and private keys.

So you want to get really under the hood? Here is how the address and WIF is calculated.

  • The interactive version is here.
  • If point 6 is exciting you, check out this Elliptic Curve visualization here

A great abstract video explaining this is below..

Code to create a new address and private key

Here is the sample code for making a new key pair.

Transaction ID’s

When bitcoin is moved from one address to another, a Transaction ID is created. A transaction ID looks like this..

053976d6471d395f51e1d4497c099926eedd3b85252d924cbc98fdd1be267d9a

With the transaction ID we can find out all the details about the transaction either from an API or from a website like Blockchain.info or directly from the blockchain via a full node.

Each transaction ID represents..

  • The unspent transactions to be used to form the transaction
  • An amount of bitcoin to send to another address
  • The current date
  • The bitcoin address that sent it,
  • The bitcoin address who should receive it,
  • The size of the transaction in bytes,
  • The fee paid to the miners for confirming the transaction
  • The bitcoin address to receive the change

The Metadata

The metadata or message that you are going to insert has a maximum size of 80 bytes. The metadata is stored in the OP_RETURN part of the transaction which was made possible in Bitcoin 0.9.

1 Character is 1 byte. The 80 bytes is enough to store a SHA1 (40 bytes) finger print of some document if you wanted plus another 40 bytes to get creative with.

Constructing a transaction

Using a ridiculously well tested library like bitcoinjs-lib version 2 or above abstracts the hashing away and makes constructing a transaction nice and easy.

NOTE: Be sure to specify the version you want as the default is old and many functions do not work as shown in the documentation.

npm install bitcoinjs-lib@2.2.0 --save

My heavily commented code below is a working starting point to make and broadcast a simple transaction.

Raw Hex

Line 41 from the code above generates the raw hex we see below. It is this hex that we broadcast.

01000000029a7d26bed1fd98bc4c922d25853bddee2699097c49d4e1515f391d47d6763905000000006b483045022100cafdaeb4be153688310d964239740dffd312ea4c480f4af9e7f79e2c8364f7f002201cfc51bea4e543278d2c19a4c29df71edcabc826b2137614a5b3ba5edc3f991d0121038a3841e2a828f2a2e6d7441c7e5d5a63b83efc167e6802541a933d7345509dd7ffffffff70aeb7b73851b03d0ff46610369444895160637d4de91c61b527f4777252683b000000006b483045022100c295505465c15d89f335c0c25ee2e408e60699198ebbf71e53ccba0bd1cc395b0220742900244f092fc4e067fe871fdde5a57198bea58c3b6ab7b4596e49887084a90121038a3841e2a828f2a2e6d7441c7e5d5a63b83efc167e6802541a933d7345509dd7ffffffff03b80b0000000000001976a914bf5d44c70abecc10f5125c3999c1baab2ebfab6988ac00000000000000000b6a094343021500607663a0b80b0000000000001976a914e2741e42426c751cf7061365210f7c195091b2d688ac00000000

Decode a raw hex such as the one above

A typical transaction is about 250 to 500 bytes in size as a raw hex

Broadcasting a transaction

You can use an API or a website to broadcast a hex transaction to the blockchain. Examples are blockchain.info/pushtx or another on blockr.io/tx/push

Depending on the fee you gave to the miners you can be waiting quite a while. As an example a 1000 Satoshi fee today took just under two hours.

Transactions don’t show on the blockchain until they are confirmed by the miners but you can view unconfirmed transactions here. Unconfirmed transaction are not guaranteed to become confirmed.

Once broadcast and confirmed you can see your message on the blockchain like the one I did here where a added my twitter handle to a transaction. Before you ask, the only reason I did this, is because I could.

Transaction Fees

To get a transaction confirmed and added to a block on the blockchain you must pay a fee to the miners. The miners need this fee to pay for mining hardware, electricity and earn some profit.

The amount remaining when the value of all outputs in a transaction are subtracted from all inputs in a transaction is the fee is paid to the miner who includes that transaction in a block.

The amount you pay for this confirmation is based on the size of the transaction measured in bytes and the speed in which you want your transaction to be confirmed, measured in blocks.

50 Satoshi = 1byte for confirmation in 1–2 blocks. Where 10 Satoshi per byte is enough for most testing.

Check todays fee estimates here

Full Node = Full Control

At points in this article I have referenced API’s and Websites for querying the blockchain, it doesn't take long before you may feel restricted by API rate limits, slow and varying response speeds and a general lack of control caused by having someone else who could turn out to be unreliable or untrustworthy between you and the blockchain.
The answer for me was to run my own full and unpruned node.

With the growing 150–200GB disk space required to host the full blockchain and all transactions, hosting options are currently limited to getting your own dedicated server or looking at Microsoft Azure.

Due to launch soon… Elastic File System from AWS and Block Storage from Digital Ocean will give some better options.

Costs are expected to be about 0.30 USD per GB pr month so prepare 60 USD per month per node for a full un-pruned with full transaction. So 3 full nodes is only going to be 180 USD per month.

If you want to just play, and production isn't a consideration then you can run a full, although pruned node in a docker container or on a raspberry pi

Parsing the blockchain into a realtime database like firebase would be very cool but cost allot more, and take some time to write the parser but it wouldn't need multiple nodes. Firebase storage costs $5/GB so a 200 GB node would bring you in at $1,000 per month then you need to add $1/GB for transfer costs. For most this isn't an option so we will leave it there.

Load Balancing your API and Full Nodes.

If you got this far you will be wanting to make sure your nodes are decentralized and resilient. And the entire setup is inline with the blockchain ethos.

The links below are for you to explore when preparing your API and nodes for production.

Still craving more blockchain nerdy treats and sustenance?

Enjoy every moment of this amazing video from Andreas…

Andreas M. Antonopoulos is a California-based information security expert, tech-entrepreneur and author. He is a host on the Let’s Talk Bitcoin podcast and a teaching fellow for the master in science Digital Currencies at the University of Nicosia.

--

--