Edge Cases: The Metanet Blog

Part 2.3 — Protocol Locality

Jack Davies
nChain
17 min readOct 16, 2019

--

Welcome back to Edge Cases, the Metanet blog series. If you’re just joining us, you can find the start of the series here.

In the last post, we looked at the types of domain structure that emerge from the properties of trees created using the rules of the Metanet protocol. We looked at how such tree structures can be mapped to domain-like constructs, and how we can interpret the permissioning hierarchy of the Metanet protocol as similar in function to the chain of trust model employed by many parts of the existing Internet.

Today, we look at how Metanet graphs interact with existing data-oriented protocols that have been developed for use on Bitcoin SV in recent months, and how they lead to a richer description of the content data embedded within such tree structures.

Part 2.3 — Protocol Locality

We begin today’s post by casting our minds back to late November 2018. It was then, at the CoinGeek Week conference, that Dr Craig Wright made the first public announcement of his vision of the Metanet.

Dr. Craig Wright’s unveiling of the Metanet project.

In order to understand exactly where the Metanet protocol sits within the thriving present-day landscape of Bitcoin SV, we first need to understand exactly how the playing field looked back then, and how it has evolved in the subsequent months.

At the time of the talk, there were a number of OP_RETURN protocols already publicly described and up-and-running, with various functions, such as the Tokenized protocol and the blockchain-based Memo platform.

The standard at the time was that such protocols were added to a central Github registry, which you can still find here. Each protocol is assigned a 4-byte protocol prefix, which is then used as a marker in an OP_RETURN transaction output to indicate that a given transaction has been created according to a particular protocol specification or certain rules.

The wider ecosystem would then refer to this registry as the central source of information when (i) trying to use an existing protocol; or (ii) trying to design and add a new protocol.

The registry, as it survives today, looks like this:

The OP_RETURN protocol registry, each protocol is identified by a 4-byte prefix.

So, how have things changed from the extant paradigms of November 2018?

Well, the concept of using short hexadecimal prefixes for certain protocols is still in full force. The Metanet protocol itself does, of course, use the protocol prefix 0x6d657461, which is just the hex representation of the word ‘meta’.

The current standards for hexadecimal protocol prefixes, used for all manner of Bitcoin SV applications, can be found in the documentation here.

Moreover, you can find a list of Bitcoin SV application protocols using this specification listed on the same github repository, which currently looks like this:

Prefix,DisplayName,Authors,BitcoinAddress,SpecificationUrl,TxidRedirectUrl0x00000020,Tokenized,James Belding,14G5z9kbACK9SiSgZnZmgpxn7aetBUD5vY,http://tokenized.cash/,0x00555354,UniSOT,Stephan Nilsson,12L7P3J6LySz2DtGS7vrrnbYAPKcZf8kiu,http://unisot.io/ust,0x00584350,Counterparty Cash,Counterparty Cash Association (CCA),17YhxKgKunLjvi7HB1ENGmeLwPFiaxb74V,https://counterparty.cash,0x00746C6B,Keyport,Keyport,,https://keyport.io,0x02446365,SatoshiDICE,Jon Bestman,1DiceuELb5GZktc3CMEv868DMtU3B5957x,https://satoshidice.com,0xac1eed88,Miner ID,Steve Shadders,,,0xff000000,extended protocol id mask,,,,0x6d6f6e6579627574746f6e2e636f6d,Money Button,Ryan X. Charles,,https://www.moneybutton.com

The hexadecimal prefix methodology is clearly still popular, and is used by a number of compelling applications on Bitcoin SV today. However, as many of you may already be aware, an alternative new paradigm has also emerged alongside the existing hex prefix method.

Bitcom Protocols

In late January 2019, a new paradigm for Bitcoin SV application protocols and development was released by _unwriter.

Bitcoin addresses as protocol prefixes.

This new paradigm, as it came with _unwriter’s Bitcom, represented an interesting shift in thinking when it comes to such protocols, and has since been widely used by developers in the Bitcoin SV ecosystem.

In short, the Bitcom concept has since become a compelling new model for creating application protocols for the Bitcoin SV blockchain, which works alongside existing protocols using the hexadecimal prefixing standard. The combination of the two methodologies has been a significant catalyst for the explosion of development we have seen on Bitcoin SV over the past year.

I would highly recommend reading through the Bitcom documentation available at the link below. It goes into the full details about how the system works, and why it may be a sensible choice in assigning protocol identifiers for many blockchain-based applications.

But, for the purposes of today’s post, the salient points of the Bitcom paradigm can be summarised as the following:

  • Bitcoin addresses (e.g. 1Rxg…) are used as protocol identifiers;
  • Protocol listing is effectively decentralised;
  • There is no requirement for a central protocol registry; and
  • Control of ownership can be provably expressed by signing with an associated key-pair.

The last point is quite profound for our vision of the Metanet as a global value network: not only do users own their data, but protocol developers can now provably control ownership of the namespace of their protocols.

In my mind at least, herein lies the most elegant part of the entire Bitcom paradigm: the fact that mechanisms for the provable control of ownership of the protocol can be implemented using the existing infrastructure of the underlying blockchain itself. The same key pairs and addresses that are used to sign transactions and transfer control of ownership of the native Bitcoin SV token can also be used to prove control of a given protocol specification, and even to sign subsequent updates and changes to its core design.

Now that we are familiar with both the general concepts of using 4-byte values for protocol identifiers and alternative using Bitcoin addresses to do the same, let’s take a look at some of the protocols that have been recently developed, and how these fit into the Metanet landscape.

The B-protocol

One of the earliest and most profound examples of a protocol created in line with the Bitcom standard was the ‘B-protocol’, also proposed by _unwriter.

Store and reference data on the Bitcoin blockchain.

This protocol does exactly what it says on the tin, if you’re willing to look a little closer at the name itself, and where its origins may lie. For those unfamiliar, a quick google of the term ‘B protocol’ will direct you to a particular file transfer protocol, developed for CompuServe many years ago.

The protocol, implemented in 1981, was designed to facilitate basic file transfer functionality. So, it makes sense to assume that the 2019 B-protocol, which was designed for the simple storage and referencing of files on the blockchain, probably takes inspiration in its nomenclature from its Internet-oriented namesake.

The B-protocol we are interested in, then, was intended to provide a standard structure and encoding for simple file types to be included as part of the data payload of an OP_RETURN transaction output.

Let’s take a look at the schema of the B protocol.

Schema:

OP_FALSE OP_RETURN
19HxigV4QyBv3tHpQVcUEQyq1pzZVdoAut
[data]
[media type]
[encoding]
[filename]

We can see that the protocol provides a simple, five-element schema for packaging file data inside an OP_RETURN transaction.

Note that we have also included such outputs as ‘OP_FALSE OP_RETURN’ transactions so as to conform with the new standard data-carrier transaction format and as intended for the upcoming Genesis upgrade to the Bitcoin SV node implementation.

The first element of the protocol is, naturally, the protocol identifier, seen here as a Bitcom-generated Bitcoin address: 19HxigV4QyBv3tHpQVcUEQyq1pzZVdoAut

The rest of the elements of the protocol schema, in order, correspond to the following:

  • [data] : the blob of data to store;
  • [media type] : the type of media that is being stored;
  • [encoding] : the encoding used for the blob of data; and
  • [filename] : the name for the file that is being stored.

It is much easier to see what they mean when looking at how these elements are populated in the example usage shown below:

Usage:

OP_FALSE OP_RETURN
19HxigV4QyBv3tHpQVcUEQyq1pzZVdoAut
<html><body>hello</body></html>
text/html
utf-8
index.html

We can make sense of this very quickly as being the B-formatting for an ‘html’ or simple ‘text’ file, encoded as ‘utf-8’ characters and named ‘index.html.’ The same file protocol is now used extensively to store such data on the blockchain. A clear example of its usage manifests itself on the bitstagram application, which renders B-encoded images, like the one shown below, in the browser.

An image on bitstagram, stored using the B protocol.

But, not only can we store such files on the blockchain as part of Bitcoin SV transactions, we may also use them in applications simply by referencing them by their TxIDs or content hashes:

Referencing by TxID:
https://bico.media/[txid]
https://bico.media/a3907e5b910f798c8d0fb450d483a0aefa5ce40ac74064b377603e5ea51deccb
Referencing by content hash:
https://bico.media/[content hash]
https://bico.media/6bfab10d3801872d6df55bb4345433e418d1ca6ffb5002be99d4d673dcaf81bc

The B-protocol for file storage and referencing is a powerful first example of application protocols built for Bitcoin SV using the Bitcom paradigm. Let’s take a brief look at some other notable examples.

The D-protocol

The D-protocol was designed for ‘dynamic’ data, which is catered for here by allowing a ‘D-transaction’ to be written to the blockchain that specifies what should be interpreted as the current state of some content data. The crucial aspect here being that the state can be overwritten by incrementing a sequence number, which is not dissimilar to how the state of a transaction can be updated in a Nakamoto payment channel.

We can see how such a scenario is implemented in the D-protocol schema and example usage shown below:

Schema:

OP_FALSE OP_RETURN
19iG3WTYSsbyos3uJ733yK4zEioi1FesNU
[key]
[value]
[type]
[sequence]

Usage:

OP_FALSE OP_RETURN
19iG3WTYSsbyos3uJ733yK4zEioi1FesNU
"zebra.jpeg"
0f9e7b6542eba739bd407323f58d75327dd1d68a53d2a535337a4dcf0ddb0edc
tx
1

In this example, the “version-1” data of the content to be known as “zebra.jpeg” resides at the transaction whose identifier is “0f9…edc,” which allows us to retrieve the data and now interpret the version of the corresponding image file.

The BCAT-protocol

The BCAT-protocol is designed to overcome the upper limit on the amount of file data that can be uploaded inside a single transaction, which, at the time of writing, is bounded by the maximum transaction size of 100kB as per the policy of the majority of mining hashrate in the BSV ecosystem.

The BCAT-protocol circumvents the limit by allowing a file to be split into several chunks, each up to 100kB in size, and mined into the blockchain by means of separate transactions. The key innovation here is that all of the ‘chunk’ transactions are linked by an on-chain anchor transaction, which specifies the transaction IDs of each of the transactions that contain a constituent part of the whole file.

The schema used for the anchor transaction looks like this:

OP_FALSE OP_RETURN
15DHFxWZJT58f9nhyGnsRBqrgwK4W6h4Up
[info]
[media type]
[charset]
[name]
[flag]
[txid part 1]
[txid part 2]
...
[txid part n]

The schema used for any of the chunk transactions is simply the now familiar B-protocol, which is used to encode the data chunk itself. So, an example schema for the chunk corresponding to ‘part 1’ of a file simply looks like this:

OP_FALSE OP_RETURN
19HxigV4QyBv3tHpQVcUEQyq1pzZVdoAut
[data]
[media type]
[encoding]
[filename]

The Author Identity Protocol (AIP)

The Author Identity Protocol (AIP) allows for explicit, selective signing of data payloads stored in OP_RETURN outputs in a highly flexible manner.

Schema:

OP_FALSE OP_RETURN
15PciHG22SNLQJXMoSUaWVi7WSqc7hCfva
[Signing Algorithm]
[Signing Address]
[Signature]
[Field Index 0]
[Field Index 1]
...

Usage:

OP_FALSE OP_RETURN
[Field 1]
[Field 2]
[Field 3]
[Field 4]
15PciHG22SNLQJXMoSUaWVi7WSqc7hCfva
BITCOIN_ECDSA
1EXhSbGFiEAZCE5eeBvUxT6cBVHhrpPWXz
0x1b3ffcb62a3b4...d268d944ee6cb0af
0
1
2
3
4

We will see in the next section that there is a missing ingredient, that we have not yet touched on regarding Bitcom protocols, that highlights the extensive utility of AIP.

The Magic Attribute Protocol (MAP)

The Magic Attribute Protocol (MAP) is a very useful protocol for mapping attributes to on-chain hyper media and content data using simple key-value pairs.

Schema:

OP_FALSE OP_RETURN
1PuQa7K62MiKCtssSLKy1kh56WWU7MtUR5
[SET|DELETE]
[key]
[value]

Usage:

OP_FALSE OP_RETURN
1PuQa7K62MiKCtssSLKy1kh56WWU7MtUR5
'SET'
'url'
'https://map.sv'

The Bitcoin Author Protocol (BAP)

The Bitcoin Author Protocol (BAP) is a simple but powerful protocol that facilitates on-chain identity management and generic data attestation. It is currently used by LegallyChained, for example.

Schema:

OP_FALSE OP_RETURN
1BAPSuaPnfGnSBM3GLV9yhxUdYe4vGbdMT
[ID|ATTEST|REVOKE]
[URN Attestation Hash]
[Sequence|Address]

Usage:

OP_FALSE OP_RETURN
1BAPSuaPnfGnSBM3GLV9yhxUdYe4vGbdMT
'ATTEST'
d4bcdd0f437d0d3bc588bb4e861d2e83e26e8bf9566ae541a5d43329213b1b13
0

Similarly to the AIP, the Bitcoin Author Protocol really comes alive when we include an additional concept that we have thus far failed to mention.

The Unix Pipe

So far, we have only talked about how each Bitcom protocol works, and the schemas used by each, in isolation. When we begin to use these protocols in conjunction with one another, they become a whole different beast.

The Unix pipeline allows for protocol interoperability.

We can employ the Unix pipeline ( | ), which is a well-known paradigm for how messages can be passed between processes in an operating framework, to pipe one Bitcom protocol ‘onto’ another.

Protocol 1 | Protocol 2 | Protocol 3 | ...

The overarching protocol effectively allows for passing the impact of one protocol on some data to the next protocol, which in turn passes on to the next protocol; so on and so forth. If we now look at this paradigm from the perspective of the Bitcom protocols we have just considered, we start to see a flow for OP_RETURN payload data that looks something like this:

OP_FALSE OP_RETURN
[Bitcom Protocol 1]
|
[Bitcom Protocol 2]
|
[Bitcom Protocol 3]
|
...

There is a very comprehensive explanation here of how we can apply the principle of the Unix pipeline to our Bitcom protocols in order to combine the disparate functionalities of many such protocols to cater for new applications and use cases for on-chain data.

The main take-away for today’s post is simply that the Bitcom protocols, that we can create separately, can be treated in a modular way. We can think of each as having a specific function, or operating on our content data in a particular way, and these modules can be combined in any way necessary to achieve a desired overall functionality.

Let’s look at how this works for a couple of examples:

D piped onto B:

OP_FALSE OP_RETURN
19HxigV4QyBv3tHpQVcUEQyq1pzZVdoAut
[DATA]
[MEDIA TYPE]
[ENCODING] or NULL
[FILENAME] or NULL
|
19iG3WTYSsbyos3uJ733yK4zEioi1FesNU
[key]
NULL
b
[sequence]

By simply piping the D-protocol (for ‘dynamic’ data) onto the B-protocol (for ‘file’ data), we create a schema for uploading explicitly versioned data to the blockchain. In other words, an interpreter could read the transaction alone and yet easily identify content data, corresponding not only to a particular file, but to a particular version of the file.

AIP piped onto B:

OP_FALSE OP_RETURN
19HxigV4QyBv3tHpQVcUEQyq1pzZVdoAut
[Data]
[Media Type]
[Encoding]
[Filename]
|
15PciHG22SNLQJXMoSUaWVi7WSqc7hCfva
[Signing Algorithm]
[Signing Address]
[Signature]
[Field Index 0]
[Field Index 1]
...

Usage:

OP_FALSE OP_RETURN
19HxigV4QyBv3tHpQVcUEQyq1pzZVdoAut
{ "message": "Hello world!" }
applciation/json
UTF-8
0x00
|
15PciHG22SNLQJXMoSUaWVi7WSqc7hCfva
BITCOIN_ECDSA
1EXhSbGFiEAZCE5eeBvUxT6cBVHhrpPWXz
0x1b3ffcb62a3b4...d268d944ee6cb0af
0
1
2
3
4
5
6

Piping AIP onto B allows us to combine simple file upload with explicit control of ownership (or attestation to control of ownership) of the file contents. The ability to explicitly sign an OP_RETURN payload in a transaction, by separating such explicit signatures from those that sign transaction inputs, is extremely useful for a number of use cases.

One example would be an application that allows users to control ownership of their data, but deals with all the funding of transactions behind the scenes, without the user having to sign with their wallet keys.

AIP piped onto BAP:

OP_FALSE OP_RETURN
1BAPSuaPnfGnSBM3GLV9yhxUdYe4vGbdMT
[ID|ATTEST|REVOKE]
[URN Attestation Hash]
[Sequence|Address]
|
15PciHG22SNLQJXMoSUaWVi7WSqc7hCfva
[AIP Signing Algorithm]
[AIP Signing Address]
[AIP Signature]

Usage:

OP_FALSE OP_RETURN
1BAPSuaPnfGnSBM3GLV9yhxUdYe4vGbdMT
ATTEST
d4bcdd0f437d0d3bc588bb4e861d2e83e26e8bf9566ae541a5d43329213b1b13
0
|
15PciHG22SNLQJXMoSUaWVi7WSqc7hCfva
BITCOIN_ECDSA
1Po6MLAPJsSAGyE8sXw3CgWcgxumNGjyqm
G8wW0NOCPFgGSoCtBT...wqbOT3Ofb1CYdNUv5a/jviPVA=

This example of protocol interoperability is powerful as it allows for the explicit signing of an attestation, which is clearly suited to applications that require evidence of an event having been witnessed, or to a document-signing platform.

Protocol Locality

Having used today’s post to go on a bit of a detour, with a fairly lengthy recap of the conventions that have emerged under the Bitcom standard, it’s time now to turn our attentions back to the Metanet protocol. The reason I have spent the time going over these conventions in some detail is to highlight one of the core properties and strengths of the Metanet protocol: protocol locality.

As I’ve already stressed in previous entries in the series, the Metanet protocol is designed to be as generic as possible, whilst still preserving the useful functionalities of both having a directed graph data structure and leveraging a well-defined flow of permissions throughout the structure.

The fact that the Metanet protocol natively supports the integration of Bitcom protocols, despite having been developed prior to their emergence, is something that I feel pays testimony to the generality I have claimed so many times already.

Bitcom protocol integration within the Metanet protocol.

The diagram above shows, in a simplified way, how such generality is the case. You can think of is as either the Metanet protocol ‘wrapping around’ a Bitcom protocol, or a Bitcom protocol ‘plugging into’ the Metanet protocol, depending on your philosophical stance. But, whichever way you want to interpret the relationship between Bitcom and the Metanet, we can all agree that they are not only compatible with one another, but they are in fact complimentary.

A Bitcom protocol does not ‘need’ the Metanet protocol to function, and likewise the Metanet protocol does not ‘need’ a Bitcom protocol to function. Yet, when they are used together they can make one another far more powerful.

What this means is that a Bitcom protocol can be developed as usual, purpose-built for a specific need, and later plugged into the Metanet protocol only if a certain use case requires structure and permissioning. The vast majority of the Bitcom protocols that have been developed thus far can be immediately integrated into a Metanet graph structure without modification.

Example: B protocol and Metanet

We can see clearly that the schema used for the B-protocol can simply fit ‘inside’ the OP_RETURN data payload underneath the elements necessary for the Metanet protocol. This is also true of virtually all Bitcom protocols we have as of the time of writing.

An ‘exception’ to prove rule: the BCAT protocol is an example of a Bitcom protocol that would need a minor adjustment to integrate directly with the Metanet protocol.

This is because BCAT has a parent transaction that forward-references its children by TxID, while the Metanet protocol has child transactions that backward-reference their parents by TxID. To have TxID references both backwards and forwards, we would create a ‘chicken-and-egg’ self-referencing problem in the TxID hash values.

All this really means is that BCAT and Metanet are trying to define the same relationship, and one duplicates the work of the other. So, a minor modification to create a Metanet-friendly version of BCAT would simply involve removing the TxID references in the anchor transaction.

But, don’t forget the point I have just made that BCAT works perfectly well outside of Metanet, and such a modified version of BCAT would only need to be made in the event that an application required large, chunked file upload that also include the structure and permissioning aspects that the Metanet protocol provides.

In general, all of our Bitcom protocols can be effectively piped onto the Metanet protocol, using the Unix pipeline ( | ) in the exact same way. This is not only true for Bitcom protocols, but also for hexadecimal protocols, and any combination thereof.

I should note that, although the technical summary document does not currently include mention of the Unix pipeline, the principle naturally applies to Bitcom protocols that are ‘piped’ onto the Metanet protocol.

This point brings us nicely onto MOM.

MOM

The Metanet Object Model (MOM) is the result of yet another leap in thinking for how we parse and handle OP_RETURN payloads containing combinations of Bitcom protocols and data. The concept it put simply by _unwriter as:

BOB + Metanet = MOM

I’m afraid to the say that there is another unknown in that equation that today’s post probably hasn’t the scope to cover: the concept of Bitcoin OP_RETURN Bytecode (BOB).

It is highly recommended that the reader defers to the in-depth articulations of BOB and MOM at their respective introductory articles here:

To put it in just a few words, BOB defines an abstract ‘cell-and-tape’ machine model for Bitcom protocol data that resides in OP_RETURN outputs, leaning heavily on the Unix pipeline, and MOM simply takes this one step further by applying the model to OP_RETURN payloads that use the Metanet protocol.

This may seem like all too much to take in at once ( — in fact, it almost definitely is), but the most important thing to take away from this is that we now have a method of modularising not only Bitcom protocols but also the Metanet protocol, allowing each to be treated and read independently while being part of the same output script.

Moreover, MOM introduces the paradigm that we can read the nodes of Metanet graph structures by expecting that the first cell or module will be an instance of the Metanet protocol schema, which can be followed by any combination of Bitcom protocol cells.

To cut a very long and abstract story short, MOM is the formalisation of a concept I have been talking about for some time known as protocol locality.

Global structure with granular content

The idea I am referring to by the term ‘protocol locality’ is simply that each and every node in a Metanet graph can utilise any local protocol, or combination thereof, to express the content of that node. The Metanet protocol, on the other hand, is used to enforce and express the global structure of the graph of nodes as a collective, rather than trying to resolve anything in particular about the content data at a particular node. Note that this fact is true whether the local protocol uses the hexadecimal prefixing standard or the Bitcom standard.

We now have two distinct ‘resolutions’ that we can use to calibrate our lenses when looking at on-chain data:

Global resolution — the over-arching structure of the Metanet graph in which a node resides.

Local resolution — the granular details of the content data situated at a particular node in the graph.

In other words, the Metanet protocol acts like a satellite image that tells you the geography of the landscape a node exists in, while local protocols act like scientists on the ground inspecting the composition of the soil at a very particular point in that landscape.

So, now when we look at a Metanet graph, instead of just seeing our nodes as blue circles with an address and a TxID, we can use MOM to resolve the details of each node in terms of Bitcom protocols.

In the specific context of local Bitcom protocols, this can be seen very clearly in the final diagram below, where each Metanet node has been decomposed in terms of Bitcom cells, than we could previously see by just looking at our graph from the bird’s eye view of the Metanet protocol alone.

A Metanet graph resolved in terms of Bitcom and MOM. (Credit: _unwriter)

It is important to remember also, that we could as easily replace each Bitcom protocol (Bitcom0, Bitcom1, Bitcom2) with a corresponding hexadecimal prefix. This achieves the same outcome, in that the local protocol in either case allows us to interpret the content of a node at a lower level, while the Metanet protocol provides a global view of how that particular node sits within both a Metanet graph and indeed the blockchain itself.

Conclusions

After a fairly lengthy post, it’s a useful exercise to summarise the key points we have made.

  1. We now have two standards for OP_RETURN application protocols: Hexadecimal prefixing and Bitcom.
  2. Application protocols defined in either way provide local resolution for interpreting each individual node in a Metanet graph.
  3. The Metanet protocol provides a global resolution that contextualises each node in a wider context.

When all is said and done, a Metanet node using a Bitcom application protocol is a wonderful illustration of how the two protocol prefixing paradigms can compliment each other to provide richer functionality in a Metanet graph.

Stay tuned for the next instalment, where we’ll be looking at the variety of transaction structures that are possible under the Metanet protocol.

Resources

The following resources will be useful as supporting material for this series in general:

  1. Metanet Protocol (Technical Summary).
  2. _unwriter’s explanation of the Metanet.
  3. Metanet slides from CoinGeek Toronto (2019).
  4. The Genesis Metanet tree: graph, thread.
  5. The MetaWriter tool: github, thread.
  6. Metanet planaria.

Copyright

© 2019 nChain Limited. All rights reserved. This article is provided without any warranties whatsoever and shall not result in the grant of any license, whether implied or otherwise. nChain Limited shall not be liable in any way for the use of the information provided herein.

--

--

Jack Davies
nChain
Editor for

Blockchain researcher, passionate about growth. Views are my own. probsnothing.substack.com