A Comprehensive Guide to RLP Encoding in Ethereum

In this blog post I will be introducing you to a method of data serialisation known as Recursive Length Prefix (RLP), which plays an important role in the implementation of Ethereum.

The goal of this post is to help you understand what exactly RLP data serialisation is, why it is used across Ethereum, and what problems it solves. We will focus mainly on RLP encoding in this post to maintain a concise scope, leaving RLP decoding for another day (and with a much shorter post!).

My opinion is that its more important to understand RLP encoding with confidence before worrying about going in the opposite direction. Regardless of the omission of decoding, gear up for a fairly dense read but prepared with much care, I hope you find it useful 🙂

To help you ease into the topic, let us get some contextual understanding around the concept of data serialisation in computer systems. You are welcome to skip forward to the next section of the post if you want to dive straight into RLP serialisation.

What is data serialisation?

Serialisation is the process of converting a data object into a series of bytes for the purposes of storage, transfer, or distribution.

Data being supplied as an input to a serialiser and producing an encoded output in the form of a stream of bytes

The serialised representation of the data (as a collection of bytes) may be used to reconstruct a semantically identical clone of the original data in another environment. Such a reconstruction describes the reverse process of serialisation known as deserialisation.

A stream of bytes being supplied as an input to a deserialiser and producing a decoded output in the form of a higher level formatted data

Computer systems can vary in hardware architecture, operating systems, and addressing mechanisms, among many other factors; with this in mind, we can understand how representation of data may vary across different systems. As a lighter analogy of this above-mentioned data representation difference, imagine how 2 fundamentally different programming languages (say JavaScript and Golang) can both represent a dictionary but fundamentally do it in different ways.

These different representations of data across systems present a challenge to how we approach the transmission and storage of data between uncommon environments. The solution to this is providing a common, standardised format that all systems communicating with each other can understand. We will refer to such standardised formats as data serialisation formats for brevity.

Many software developers are probably already familiar with some data serialisation formats such as XML and JSON, both of which are commonly used in many applications.

How data serialisation and deserialisation works

A data payload can take various forms. It could simply be a primitive type such as an integer, boolean, or string value, or in most cases, it is a composite type such as data structures like arrays, tables, trees or classes etc.

When we want to store or transmit a data payload to another location (typically over a network), we need to make use of serialisation. What serialisation does is encode the data payload into a sharable format that can be transmitted elsewhere where it can be reconstructed in a new location according to the rules of data representation in that environment.

Lets look at a simple example that makes sense of this explanation given above:

Our example is as follows; we have 2 apps communicating with each other as part of a system, we want to send a data payload from our JavaScript app to our Golang app:

Our system where our JavaScript app is sending a payload to our Golang app

Our payload is described by the UML below:

Our “Chris Redfield” data payload (a simple object)

We are going to use JSON as our data serialisation format shared by both our apps. With this in mind, our JavaScript app will do the following:

1. Define the data object.
2. Serialise the data object using JSON.
3. Send the serialised payload to the Golang app.

We can show our JavaScript app serialising our payload and sending it to our Golang app with the code below:

Our Golang app is going to expect to 1) receive serialised data and 2) deserialise it into a variable called data . We will show this with the code below:

As you can see from above, our JavaScript code serialises the data payload using the JSON.stringify() method. Our Golang code, after receiving the serialised data, is able to deserialise it into a variable using the json.Unmarshal() method.

And thats the example. We have just observed a very basic form of data serialisation between 2 applications with fundamental differences in data representation! We will actually see what this kind of data payload looks like when expressed as a sequence of bytes in the next section of this post, so stay tuned.

That concludes our brief background introduction to data serialisation. Now let us move onto questions regarding where data serialisation is utilised uniquely in Ethereum.

Data Serialisation in Ethereum

Ethereum can be described as a decentralised network comprised of nodes (computers) that run Ethereum clients (software implementations of the Ethereum protocol specification) which communicate with each other in peer-to-peer fashion.

Without diving into the details of how Ethereum works (I will cover this in a future blog post), there is one important thing to understand before reading ahead:

Ethereum creates, stores and transfers large volumes of data.

Why is this piece of information relevant to us?

It is relevant to us and our interest in Ethereum because large storage and bandwidth requirements pose a challenge to running a node on consumer-grade hardware, of which is a crucial requirement for maintaining a truly resilient, censorship-resistant, decentralised network. If we cannot maintain a healthily decentralised network, we lose the valuable properties of a blockchain that justifies its value proposition.

To help reduce the costs of running a node, the Ethereum community made optimisations in the form of standardised data formatting to reduce storage and bandwidth requirements in memory-efficient manner. To achieve this, several specific data structures (we will not look at these in this post) and data serialisation formats are used on the Ethereum stack.

In Ethereum there are actually 2 dominant formats that are utilised:

1. Recursive Length Prefix (RLP)
2. Simple Serialize (SSZ)

We will not be going over SSZ at all in this post, but if you would like to know more there is a good article for it here.

Recursive Length Prefix (RLP)

Recursive Length Prefix (RLP) is a data serialisation scheme used heavily in Ethereum execution clients that help standardise the transfer of data between nodes in a space-efficient format.

Execution clients manage tasks like transaction execution and transaction gossip.

We can formally define RLP as a serialisation method for encoding arbitrarily structured binary data (byte arrays) while leaving encoding of primitive data types (e.g. strings, floats etc) up to higher-order protocols.

An important detail to understand about RLP encoding is that the data inputs are expected to be represented as arrays (or nested arrays) of binary data in order to be valid inputs to be serialised.

What exactly does representation as an array of binary data mean?

Well imagine we want to serialise an object (or dictionary for the Python folk) that is equal to{ k1: v1, k2: v2 } .

As it stands, this is not a valid input that can be serialised via RLP.

A recommended format change would be to describe the object as an array of key-value tuples with the keys in lexicographic order. With this description we can describe our object as [[k1, v1], [k2, v2]] , which is now a valid input for RLP encoding.

The variables k1 , v1 , k2 , and v2 represent binary data payloads (or nested arrays of binary data payloads depending on how complex the original data structure actually is).

For example imagine k1 is equal to a single character 'A' and v1 is equal to the string "foo" ; k1 as a binary payload would be equal to a byte with value 65 and v1 would be equal to a byte array [102, 111, 111] .

NOTE: Do not confuse these byte representations of the variables with actual RLP encoding (of which we have not even defined!). I am simply showing you what individual values of a primitive key-value pair look like when expressed as bytes.

It would be wise to remember that a single byte represents a value in the integer range [0, 255] which can also be expressed in hexadecimal form as [0x00, 0xff] . This is extremely important to understand before we move onto the definition of RLP serialisation.

With the above being said, let us just take a step back and get a solid understanding for how we represent strings as a collection/array/sequence of bytes (a.k.a a byte array). This is something that will really help you practically understand how we RLP encode data. We will also see why some serialisation schemes like JSON are not suitable for serialising data across Ethereum.

Representing strings as byte arrays (sequences of bytes)

Imagine we have the JSON string, "{"name":"Chris Redfield","age":30,"job":"STARS soldier"}" .

This string can be represented as a sequence of bytes, where each ASCII character in the string can be converted to a byte value ranging from 0 to 255 (in decimal representation) or 00 to FF (in hexadecimal representation). As we move forward in this post, we will be dealing with bytes under hexadecimal representation exclusively to avoid any confusion.

So with this knowledge in mind, let’s just see what our above string would look like if expressed as a sequence of bytes under decimal representation and hexadecimal representation respectively:

Expressing a string as byte arrays under both decimal and hexadecimal representations

So you might be thinking now, “JSON seems like a pretty good option for managing data serialisation in Ethereum”. Let’s briefly explore why it is in fact quite the opposite, especially in the context of blockchains and specifically Ethereum.

JSON serialisation DOES NOT guarantee deterministic outputs across implementations, see here for a brief but insightful answer as to why JSON doesn’t guarantee this. If you are lazy to read, it came down to failures to standardise this property.

In the context of blockchains and specifically Ethereum, this poses an immediate problem because serialised data plays a role in computing several important hashes such as transaction hashes and block hashes. If these hashes are not deterministic, we end up transactions and blocks corresponding to multiple hashes. You can imagine from here verification problems just get worse, especially in a peer-to-peer network such that requested merkle proofs do not compute the correct merkle roots.

The bottom line is that we need a serialisation scheme that produces deterministic outputs.

RLP Encoding Algorithm Explored

With RLP encoding, we receive an input in the form of an item, which is fed into a serialisation algorithm to produce an encoded output in the form of RLP-serialised data.

An item is defined as a:

• String (i.e. byte array representation of the string)
• List of items (i.e. list of lists, list of items, combinations of both)

The RLP encoder receives an item as the input and generates a serialised data output

Lets look at some valid items that a RLP encoding algorithm accepts:

• 0x71 (a byte)
• "a" ( a string)
• "cat" (a string)
• "" (an empty string)
• [] (an empty list)
• ["cat", "dog"] (a list containing multiple strings)
• ["cat", "dog", ["cow", "chicken"], [[]], [""]] (a list containing nested items)
• [["name", "Claire Redfield"],["location", "Raccoon City"]] (a list containing nested items, representing an object formatted for serialisation)

Notice here how the examples given are exclusively byte, string and list data types, or a combination of them. This is to emphasise that the RLP specification mainly focuses on these data types.

Non-Standardised Features added to RLP Encoding Implementations

You may find implementations of RLP encoding algorithms in various programming languages allow for types like pointers, boolean values, big numbers etc to be accepted as inputs to be encoded.

It is important to stress that this IS NOT part of the standardised RLP specification that serialises data that is transferred across different environments (e.g. A go-ethereum client gossiping RLP-encoded data to a Nethermind client).

The reason why say go-ethereum may implement RLP encoding with additional types of inputs it can accept is because it likely makes use of this serialisation internally within the client itself or with exclusive gossip between different go-ethereum clients (this is speculation on my part, but intuitively it does somewhat make sense).

So I felt it necessary to bring this up because not all test cases may be applicable across different RLP encoding implementations.

For example, the go-ethereum RLP test case inputs may be invalid as an input for the encoding algorithm in the ethereumjs RLP package in JavaScript.

What is important is that you only test inputs corresponding to the conditional cases we are about to describe below.

So with that at mind, lets discuss some of these inputs listed above:
As you can judge from some of nested item examples, representation of data in item form (combinations of lists and strings) can be quite overwhelming when we start to think about how we need to transform complex data structures like nested objects or trees into valid inputs for encoding.

Fortunately, by formatting the data in such a way, RLP algorithms are able to successfully serialise data of arbitrary structure complexity in an efficient and predictable manner. How this is achieved is by following a set of conditional cases which perform different serialisation tasks based on the characteristics of the input supplied.

Lets look at these RLP encoding algorithm conditional cases:
A valid input will always satisfy one of these conditional cases, where the input supplied is either:

• (1) A non-value (null , ’’ , false)
• (2) An empty list ([])
• (3) A single byte with value within range [0x00, 0x7f]
• (4) A string with length of 1 byte
• (5) A string with length between 2–55 bytes
• (6) A string with length greater than 55 bytes
• (7) A list with the sum of its RLP-encoded contents being between 1–55 bytes
• (8) A list with the sum of its RLP-encoded contents being greater than 55 bytes

We will now dive into what is explicitly done in each of these conditional cases to show how they serialise data differently based on their different requirements.

All outputs produced from these cases are represented as byte arrays, even if there is only 1 value calculated as part of the encoding.

Before we move on, I would like to explain one crucial detail about dealing with strings (relevant to cases 4,5, and 6):

When we are encoding each character of string, we covert the ASCII character to its corresponding character code.

Example: The letter 'r' becomes the character code with value 114 in a hexadecimal byte array this single letter would correspond to 0x72 (this is the hexadecimal representation of the character code for the letter 'r' )

Case 1

Rule: Input p is a non-value (null , ’’ , false ):

Therefore, RLP Encoding = [0x80]

EXAMPLE
Let p = '’ (empty string ε)

∴ RLP Encoding = [0x80]

Case 2

Rule: Input p is an empty list([]):

Therefore, RLP Encoding = [0xc0]

EXAMPLE
Let p = [](empty list)

∴ RLP Encoding = [0xc0]

Case 3

Rule: Input p is a single byte where p ∈ [0x00, 0x7f] :

Therefore, RLP Encoding = [p]

EXAMPLE
Let p = 0x2a ,
0x2a∈ [0x00, 0x7f], which means we satisfy Case 3

∴ RLP Encoding = p = [0x2a]

Case 4

Rule: Input s is a string that is 1 byte in length:

Therefore, RLP Encoding = [s]

EXAMPLE
Let s= [0x74] (s is the hex byte array of the string “t”)

∴ RLP Encoding = s = [0x74]

Case 5

Rule: Input s is a string that is 1–55 bytes long:

1. len = length of s
2. Calculate the first byte f:
   f = 0x80+ len | where f ∈ [0x81, 0xb7] (minimum length of string is 1)
3. Therefore, RLP Encoding = [f, ...s]

EXAMPLE
Let s = [0x64, 0x6f, 0x67] (s is the hex byte array of the string “dog”)

len = 3 (3 elements in the byte array)
f = 0x80 + 3 = 0x83

∴ RLP Encoding = [f, ...s]= [0x83, 0x64, 0x6f, 0x67]

Case 6

Rule: Input s is a string that is more than 55 bytes long:

1. len = length of string (expressed as hexadecimal)
2. Calculate bytes required to store len, b:
   b = bytes required to represent length of string in hex
   * for example, storing a length of 0xffff, requires 2 bytes since it would be expressed in bytes as len =[0xff, 0xff]
3. Calculate the first byte f:
   f = 0xb7+ b| where f ∈ [0xb8, 0xbf] (minimum length of string is 1 )
4. Therefore, RLP Encoding = [f,...len, ...s]

EXAMPLE
Let s = [0x4c, ..., 0x6c, 0x69, 0x74] (s is the hex byte array of the string “Lorem ipsum dolor sit amet, consectetur adipisicing elit”) which is 56 bytes long.

s has a length of 56 bytes, if we convert this length to hexadecimal we get 1 byte value:
len = [0x38]
* remember from our theory that if the length was a larger value like for example 0x1234 (4660 bytes) , our len value would be [0x12, 0x34] (2 bytes required to encode the length). Im only mentioning this here because I know how confusing this can be at first.

We can represent the hex length value len with a single byte [0x38] .
Therefore, the amount of bytes required to represent len is 1:
b = 1

We can calculate the first byte now:
f = 0xb7 + b = 0xb8

∴ RLP Encoding = [f,...len, ...s] = [0xb8, 0x38, 0x4c, ..., 0x6c, 0x69, 0x74]

Case 7

Rule: Input l is a list which has a payload of 1–55 bytes:

1. len = Length of each RLP encoded list item summed together (expressed as hexadecimal)
2. Calculate the first byte f:
   f = 0xc0+ len| where f ∈ [0xc1, 0xf7] (minimum length of list is 1)
3. concat = concatenation of RLP encodings of list items
4. Therefore, RLP Encoding = [f, ...concat]

EXAMPLE
Let l = ["cat", "dog"] .

"cat" can be represented as a byte array [0x63, 0x61, 0x74]
"dog" can be represented as a byte array [0x64, 0x6f, 0x67]

RLP encoding of "cat" =[0x83, 0x63, 0x61, 0x74] (length of 4)
RLP encoding of "dog" =[0x83, 0x64, 0x6f, 0x67](length of 4)

We can now calculate concat and len:
concat = [0x83, 0x63, 0x61, 0x74, 0x83, 0x64, 0x6f, 0x67]
len = 4 + 4 = 0x08

We can now calculate the first byte:
f = 0xc0+len = 0xc8

∴ RLP Encoding = [f, ...concat]= [0xc8, 0x83, 0x63, 0x61, 0x74, 0x83, 0x64, 0x6f, 0x67]

Case 8

Rule: Input l is a list which has a payload greater than 55 bytes:

1. len = Length of each RLP encoded list item summed together (expressed as hexadecimal)
2. Calculate bytes required to store len, b:
   b = bytes required to represent len
   * for example, storing a length of 0xffff, requires 2 bytes since it would be expressed in bytes as len = [0xff, 0xff]
3. Calculate the first byte f:
   f = 0xf7+ b| where f ∈ [0xf8, 0xff] (minimum length of list is 1)
4. concat = concatenation of RLP encodings of list items
5. Therefore, RLP Encoding = [f, ...len, ...concat]

EXAMPLE
I’ve written so much so far so might as well walk through a practical example seeing as people seldom provide one 🙃:
Let l = ["apple", "bread", "cheese", "date", "egg", "fig", "guava", "honey", "ice", "jam", "kale"] .

"apple" can be represented as a byte array [0x61, 0x70, 0x70, 0x6c, 0x65]
"bread" can be represented as a byte array [0x62, 0x72, 0x65, 0x61, 0x64]
"cheese" can be represented as a byte array [0x63, 0x68, 0x65, 0x65, 0x73, 0x65]
"date" can be represented as a byte array [0x64, 0x61, 0x74, 0x65]
"egg" can be represented as a byte array [0x65, 0x67, 0x67]
"fig" can be represented as a byte array [0x66, 0x69, 0x67]
"guava" can be represented as a byte array [0x67, 0x75, 0x61, 0x76, 0x61]
"honey" can be represented as a byte array [0x68, 0x6f, 0x6e, 0x65, 0x79]
"ice" can be represented as a byte array [0x69, 0x63, 0x65]
"jam" can be represented as a byte array [0x6a, 0x61, 0x6d]
"kale" can be represented as a byte array [0x6b, 0x61, 0x6c, 0x65]

RLP encoding of "apple" =[0x85, 0x61, 0x70, 0x70, 0x6c, 0x65] (length of 6)
RLP encoding of "bread" =[0x85, 0x62, 0x72, 0x65, 0x61, 0x64](length of 6)
RLP encoding of "cheese" =[0x86, 0x63, 0x68, 0x65, 0x65, 0x73, 0x65](length of 7)
RLP encoding of "date" =[0x84, 0x64, 0x61, 0x74, 0x65](length of 5)
RLP encoding of "egg" =[0x83, 0x65, 0x67, 0x67](length of 4)
RLP encoding of "fig" =[0x83, 0x66, 0x69, 0x67](length of 4)
RLP encoding of "guava" =[0x85, 0x67, 0x75, 0x61, 0x76, 0x61](length of 6)
RLP encoding of "honey" =[0x85, 0x68, 0x6f, 0x6e, 0x65, 0x79](length of 6)
RLP encoding of "ice" =[0x83, 0x69, 0x63, 0x65](length of 4)
RLP encoding of "jam" =[0x83, 0x6a, 0x61, 0x6d](length of 4)
RLP encoding of "kale" =[0x84, 0x6b, 0x61, 0x6c, 0x65](length of 5)

We can now calculate concat and len:
concat = [0x85, 0x61, 0x70, 0x70, 0x6c, 0x65, 0x85, 0x62, 0x72, 0x65, 0x61, 0x64, 0x86, 0x63, 0x68, 0x65, 0x65, 0x73, 0x65, 0x84, 0x64, 0x61, 0x74, 0x65, 0x83, 0x65, 0x67, 0x67, 0x83, 0x66, 0x69, 0x67, 0x85, 0x67, 0x75, 0x61, 0x76, 0x61, 0x85, 0x68, 0x6f, 0x6e, 0x65, 0x79, 0x83, 0x69, 0x63, 0x65, 0x83, 0x6a, 0x61, 0x6d, 0x84, 0x6b, 0x61, 0x6c, 0x65]

Since the length of concat is 57 bytes (just count the above RHS), we see that converting this length to hexadecimal results in 0x39 . To represent this value in bytes we only require 1 byte therefore the bytes to represent the length of concat is represented as:

len = [0x39] (a single byte)
* remember from our theory that if the length was a larger value like for example 0x1234 (4660 bytes) , our len value would be [0x12, 0x34] (2 bytes required to encode the length). Im only mentioning this here because I know how confusing this can be at first.

We can represent the hex length value len with a single byte [0x39] .
Therefore, the amount of bytes required to represent len is 1:
b = 1
We can now calculate the first byte:
f = 0xf7+len = 0xf8

∴ RLP Encoding = [f,...len ...concat]= [0xf8, 0x39, 0x85, 0x61, 0x70,..., 0x61, 0x6c, 0x65]

With all the conditional cases explored, we have all the building blocks needed to create an RLP encoding algorithm to encode any valid input of arbitrary complexity!

As you can see, conditional cases associated with lists will require performing encoding of its contained items to effectively serialise the list in question. From this you can deduce that lists can be holding even more list elements that we would also have to serialise within a new execution scope and so you can start to see why our encoding algorithm will require use of recursion and a divide and conquer strategy. With all our conditional cases defined and explored, this will actually be less challenging than you think!

So now you can see where the term “recursive” comes from in the context of RLP serialisation. Lets see what an implementation of an RLP encoder algorithm looks like.

RLP Encoding Algorithm Implementation

Alright, the time has come; let us finally see an implementation of this encoding algorithm.

In short here is the overly-simplified recipe:

1. The algorithm receives the input.
2. It checks the input properties against all the conditional cases described earlier on (it also checks for some extra cases which are NOT part of the standardised specification but are handy to have, I will explain these later on).
3. Based on the conditional case satisfied, the appropriate encoding process is applied. Depending on the input, there may be recursion involved in the encoding process.
4. Receive RLP encoded output 🏆

Let us see an implementation of this encoding algorithm in TypeScript:

Let’s dive into the implementation of the encoding algorithm.

If you have a glance at the code snippet above, you will notice that all our conditional cases are labelled with comments. All these labelled cases were explored earlier in this post and showed how we trivially solve for respective RLP encoding outputs.

If you look at each conditional case that is labelled (1) to (8), you will observe that we return from the encode() function call every time with a Uint8Array data type. This data type is what is used to define a byte array in JavaScript (you could also make use of the Buffer type to do so but I feel Uint8Array is a more elegant type to work with in the scope of this algorithm)

One important thing to note about the use of Uint8Array data types is that each element of the byte array becomes a decimal byte value in the range of [0, 255] . So if you think about it, you can deduce that:

1. Hexadecimal values get converted to decimal values (in JavaScript we can model such a conversion as decimal_value = parseInt(hex_value, 16) ).
2. Character codes get converted to decimal values (in JavaScript we can model such a conversion as decimal_value = (character_code).charCodeAt(0) ).

Lets see this in action with 2 brief examples:

Example 1:
Remember that the RLP encoding of the string "dog" was calculated to be[0x83, 0x64, 0x6f, 0x67] in our conditional case example. If we were to actually supply "dog" as an input to our encode() function it would actually generate an output of Uint8Array(4) [131, 100, 111, 103] .

Example 2:
Remember that the RLP encoding of the string "t" was calculated to be [0x74] in our conditional case example. If we were to actually supply "t" as an input to our encode() function it would actually generate an output of Uint8Array(1) [116] .

If you simply convert each of these output byte values from decimal to hexadecimal, we will get the byte arrays represented under hexadecimal. e.g. in Example 2, decimal byte value of 116 converted to hexadecimal is equal to the hexadecimal value of 74

Lets take a step forward now and examine how exactly nested inputs are dealt with by the algorithm and how exactly they encode structure.

Recursion

At line 111 of the code snippet you are able to see the part of the encode() function which deals with list inputs.

As you might remember from the conditional cases that walked through encoding a list and its contents, we have to first calculate the RLP encoding of all of the list elements before we can fully provide a RLP encoded output to the list itself. The reason for doing this is because we:

1. need to RLP encode each item to build the bulk of the encoded output.
2. need to find the total amount of bytes that the concatenation takes up. This concatenation refers concatenating each list item’s RLP encoding output together. We require the byte length of this concatenation to determine what sort of additional prefix byte may need to be prepended to the concatenated payload.

With this explanation, you can see how we iterate over lists to perform encode() function calls for each individual list item therefore demonstrating recursion in practice until a trivial encoded output is returned corresponding to one of our conditional cases (or add-on cases which I will explain shortly!).

Helper Functions for the Encoding Algorithm

I purposely did not explain about these minor custom functions used by our encode() function because it is not critical to grasping the encoding algorithm, but it does play a role for abstracting tedious code. If you want to dive in or step through the code, you can add these methods to whatever execution script you attach your debugger/session to:

Add-On Cases (not needed but nice to have if this would be the only implementation used across your system)

These are not needed to fulfil the core promises of the RLP specification, but are nice to have if for example you will only be using this algorithm and its decoding counterpart in a system which only has the option of using this specific implementation.

I’ll briefly list the add-on cases addressed:

• Encode hexadecimal escape sequences (e.g. '\x04\00' )
• Encode any decimal whole number (e.g. 10000 , 0 )
• Encode boolean values (e.g. false , true )

What influenced the design of this encoding algorithm?

The key things to takeaway from this dive into the internals of the RLP encoding algorithm is:

• We calculate 1 or 2 prefix bytes that essentially describe the shape and length of the data being encoded.
• The reason we make use of byte ranges in determining our first prefix byte is to help us distinguish not only the type of data that was encoded but also its length. By encoding data in such a way, we save memory space.
• While our encoding is memory-efficient, this does come at a minor tradeoff where decoding will require some context about the type of data that was encoded (we will explore this in a future blog post).

RLP Encoding in use in Ethereum

We have now reached the penultimate section of this post, where we unify our new RLP encoding knowledge with our knowledge of Ethereum and blockchains and get to see a brief example of RLP serialisation in action!

Let us see how an Ethereum block header can be computed (or verified) using RLP encoding of specific fields of a block.

Computing (or verifying a block header)

In order to compute a block header (specifically referring to the block hash), we need to:

1. Create a tuple of specific block data.
2. RLP encode this tuple.
3. Hash the RLP encoded data using the keccak256 cryptographic hashing function.

The format of this tuple described in the first step will be different for blocks that were mined before the EIP-1559 upgrade compared to blocks mined after this point. We will refer to these different block types as:

• Legacy Blocks (for blocks before EIP-1559 upgrade)
• 1559 Blocks (for blocks after EIP-1559 upgrade)

For Legacy Blocks:

Tuple = [ parentHash, sha3Uncles, miner, stateRoot, transactionsRoot, receiptsRoot, logsBloom, difficulty, number, gasLimit, gasUsed, timestamp, extraData, mixHash, nonce]

For 1559 Blocks (simply add the newbaseFeePerGasto the end of the tuple):

Tuple = [ parentHash, sha3Uncles, miner, stateRoot, transactionsRoot, receiptsRoot, logsBloom, difficulty, number, gasLimit, gasUsed, timestamp, extraData, mixHash, nonce, baseFeePerGas]

I have scraped 1 block for each of these types (and filtered out the transactions property, as it is not strictly required to calculate the block header if we have the transactionRoot )

In a future post we can dive into how we calculate this transactionRoot using knowledge of transaction envelopes, some more RLP serialisation and some merkle patricia tries 🙂 This will allow us to verify the transactionRoot .

Moving on, here are our sample blocks:

Legacy Block (Block Number = 12 964 874)

1559 Block (Block Number = 15 122 367)

From the scraped block data, we can see that:

• The Legacy Block has a block hash of 0x14687ccc4adfd2b0f0d530fc8f8c64dd29ac301ddea26e99087e11d0eae4f575
• The 1559 Block has a block hash of 0x01176f73081379cfd9d8bcee1c7e822b66f4d71160900d7f963619c42716cd83

We want to RLP encode a formatted tuple of specific block data, so lets first write a function to generate this type of tuple whenever we receive a block:

The LONDON_HARDFORK_BLOCK is just a constant for the block number 12965000 where EIP-1559 became active, introducing the block base fee.

From the above you can see that we can convert any block received, into the tuples we require to compute the block headers (regardless of block type).

The following tuples would be generated from our above 2 blocks using prepareBlockTuple():

From here we are able to make a function that will accept our formatted block tuples as an input and RLP encode them. Thereafter we can generate the block hash by hashing the RLP encoded data using the keccak256 cryptographic hashing function:

Notice here that I derive a Buffer from our previously defined encode() function, this is because the specific keccak library I am using requires a Buffer input.

We can put all the above together to create a verifyBlockHeader() function that accepts any block and will verify the integrity of the hash by forming a block tuple, RLP encoding it, and then hashing it. We compare our computation result with what was provided as the block hash when we received the block.

And there you have it, a simple but useful example of how to use RLP to calculate/verify any block header.

Conclusions

This post ended up being a lot longer than I initially planned it to be, but I am happy I got around to finishing it. I am very aware that the encode() implementation can be far better optimised and I have seen many implementations but with the way we approached defining the algorithm, I believe there is some value to introducing first time learners to these concepts in this manner.

I am by no means an expert on any of this, so if you have any feedback or corrections to point out, please feel free to do so 👌🏾

I hope you managed to learn something new from this post whether it be getting an introduction to RLP serialisation, facts about Ethereum or general computer science and algorithm-related knowledge. I apologise for any bad writing, this is my first attempt at technical writing 🙃

Going forward there are many topics I hinted at covering in future posts, so if you would like to learn more about Ethereum and the computer science around it, please like, follow, and stay tuned for hopefully improved content!