A (Practical) Walkthrough of Smart Contract Storage

Published in

Coinmonks

11 min readJul 13, 2018

Before starting, there are two fantastic sources already on the subject that cover most basic use cases quite well:

This one is great but is written in solidity which means that it isn’t incredibly easy to get your hands dirty and test:

Understanding Ethereum Smart Contract Storage

Ethereum smart contracts use an uncommon storage model that often confuses new developers. In this post, I’ll describe…

programtheblockchain.com

The second one is written in javascript but is very brief and without any tests:

How to read Ethereum contract storage

Everybody is talking that data in contracts are public, but not everybody knows how to read it.

medium.com

The aim of this tutorial is to take a more test-driven approach using truffle in order to fully see how these different data structures are stored. We will also be creating some neat tools along the way.

Discover and review best Blockchain softwares

This tutorial will not go through every possible method of storage, it will go through a PausableToken which covers many, but not all cases. One type worthy of mentioning is arrays. We will find and test for the following types:

booleans
uints
strings
mappings
nested mappings
addresses

If you get lost along the way, the complete code for this tutorial can be found here:

TovarishFin/storage-coin-tutorial

storage-coin-tutorial - code for storage tutorial found at https://medium.codylamson.com

github.com

With that being said… lets get started.

Project Setup

Install truffle globally if you haven’t already:

yarn global add truffle

Create the directory and enter it:

mkdir contract-storage && cd contract-storage

Start a truffle project:

truffle init

Let’s create the contract we are going to be using for testing:

touch contracts/StorageCoin.sol

We are going to use an ERC20 token from OpenZeppelin for some easy to use storage which might be a bit more relateable:

yarn add openzeppelin-solidity

Writing the Contract

Lets create a simple example using PausableToken from OpenZeppelin. Paste the following into contracts/StorageCoin.sol:

If you are unfamiliar with PausableToken, don’t worry. We will be taking a closer look at it and any contracts it inherits from in the next section.

This contract will do nicely since we will have the following data types to work with:

string name & symbol
uint totalSupply_ NOT decimals YES private totalSupply_
bool paused
address owner
mapping balances
nested mapping allowance

Item two above has some interesting side notes:

The reason that decimals is not going to be accessed by storage is because it does not live there! constant variables live in code and not in storage. This allows one to do some very cool tricks with this which I will get into later.

The reason that we will be accessing private totalSupply_ is because, well, it is possible. private variables are completely accessible to anyone who knows a bit about storage. To be clear, this means that you shouldn’t be assuming anything is inaccessible to users when marked private.

Preparing to Write the Tests

At this point it is worth mentioning how storage works. Contract storage essentially acts like a mapping of bytes32 to bytes32. The key can easily be interpreted as a number converting bytes to decimals:

Values are stored sequentially starting with 0:

0x00000000000000000000000000000000000000000000000000000000000000000

and going all the way up to ²²⁵⁶ or 1.157920892E77:

0xffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff

To give the last number some more context… this is an incredibly huge number. There are less atoms on earth than this number.

Storage is set sequentially in order of appearance in the contract code. All storage is stored in bytes which means it might be a good idea to check out how ascii works (strings are encoded in ascii not unicode).

With this is mind, lets find out in what order variables appear in our contract, including inheritance.

We are going to want to globally add truffle-flattener in order to better understand where storage is getting stored. This package turns a contract inheriting from multiple contracts from different files into a single file. It is also super useful for verifying contracts on etherscan.

yarn global add truffle-flattener

Let’s flatten the contract we made to see what we are working with here:

truffle-flattener contracts/StorageCoin.sol > StorageCoinFlattened.sol

Open up StorageCoinFlattened.sol in your root directory. You should have the same as this:

Writing the Tests for Simple Storage

Lets create the testfile:

touch test/StorageCoin.js

Here is our basic layout for Storage.js:

Now that we have the layout, lets create our first tool! We are going to create a function which will scan a specified number of storage slots for any values.

Create a new helpers.js file:

touch test/helpers.js

Lets add our old friend, BigNumber and chalk to make things prettier:

yarn add "git+https://github.com/frozeman/bignumber.js-nolookahead.git" chalk

Truffle uses an older version so we must make sure to install the correct version.

Now open up test/helpers.js and let’s create our first helper function:

This function will keep looking for data in storage slots starting from the first until it finds only empty slots 10 times in a row. The function that is doing most of the work here is web3.eth.getStorageAt(). This is our way of directly accessing storage from a deployed smart contract. This will also work for contracts on mainnet or any testnets.

Finally, lets put this function to use. Update StorageCoin.js like so:

We have included the helper function and included chalk and util. chalk is just for colors. util is needed in order to log out objects correctly.

Run truffle test in to see what we get for storage. You should get something like this (but with pretty colors):

local variables:
totalSupply: 0x56bc75e2d63100000
name: 0x53746f72616765436f696e
symbol: 0x535443
owner: 0x627306090abab3a6e1400e9345bc60c78a8bef57
storage:
[ { slot: 0, data: '0x0' },
  { slot: 1, data: '0x056bc75e2d63100000' },
  { slot: 2, data: '0x0' },
  { slot: 3,
    data: '0x01627306090abab3a6e1400e9345bc60c78a8bef57' },
  { slot: 4,
    data: '0x53746f72616765436f696e000000000000000000000000000000000000000016' },
  { slot: 5,
    data: '0x5354430000000000000000000000000000000000000000000000000000000006' },
  { slot: 6, data: '0x0' },
  { slot: 7, data: '0x0' },
  { slot: 8, data: '0x0' },
  { slot: 9, data: '0x0' },
  { slot: 10, data: '0x0' },
  { slot: 11, data: '0x0' },
  { slot: 12, data: '0x0' },
  { slot: 13, data: '0x0' },
  { slot: 14, data: '0x0' } ]

Lets try to decipher this and see what is what here.

Slot 0 is a space reserved for the balance mapping. We will get into that more shortly.

If we look at slot 1, we can see that this is equal to 100e18 if we paste that hex value into this tool. This is our totalSupply. We know this is totalSupply because mappings are stored differently and is already in slot 1.

Slot 2, like slot 0, is another mapping. This is the nested allowed mapping. We will also dive into this later.

We can tell immediately that slot 3 contains the owner address. But there is something else in there… this is the paused boolean. true will be 0x01 and false will be 0x00. Solidity will try to pack more than one item into a storage slot when they can be fit. This is exactly what has happened here.

Slot 4 can be figured out once pasting the value into a hex to ascii converter. This is the name variable. A keen eye might notice the 16 at the end. When converting hex to decimal, you can see that this is 22. 22 is the length of the data in the string (without 0x prefix).

'53746f72616765436f696e'.length === 22

Slot 5 is the same situation as slot 4. When converting to ascii, you will find that the value is STC, our symbol variable. You can also see that the length of the data is 6, the same as the value at the end.

Storage Stored in Order

You might be wondering at this point… I thought these were stored in order? This is still the case… but if you actually go through the inheritance of each contract you will find that the order is different. Open up your openzeppelin-solidity directory in node_modules and you will see that the order is like this: ERC20/Basictoken -> StandardToken -> PausableToken -> StorageCoin

You will see that BasicToken, the first in line for inheritance has balances followed by totalSupply. Then StandardToken has allowed.

PausableToken inherits from Pausable which inherits from Ownable resulting in paused and owner coming next in line.

Lastly the variables that we set in StorageCoin are added. name comes first be cause it is declared first and is followed by symbol. Remember that constants are not stored in storage but in the deployed code itself. So decimals is nowhere to be found in storage.

Let’s change the test block into a set of real tests:

We also need the helper which is used in the last two tests. Put this into your helpers.js file. Make sure to export it as well:

This function is returning the ascii version of the hex value from storage. We need to remove the trailing empty bytes as well as the length declaration at the end. We can do this by taking the last 2 characters (the length) and using that to state where the slice should end (the length of the string).

Understanding Mapping and Nested Mapping Storage

Mappings are different from the simple types we have looked at so far. When a mapping is declared, space is reserved for it sequentially like any other type, but the actual values are stored in a different slot. In order to find the slot you must take the sha3(keccak256) hash of the slot concatenated with the key.

We can illustrate this point with the helper that we will build for getting a mapping value.

First, we are going to need another dependency:

yarn add left-pad

Then make sure to require it at the top in helpers.js:

const leftPad = require('left-pad')

After, put these functions into your helpers.js file and export getMappingSlot:

Let’s look at getMappingSlot. We are taking both the slot and key and turning them into hex (if not already) and left padding with empty bytes. Both of them need to be in what is essentially bytes32. We are padding to 64 because each byte is two characters.

We then create a hash from the formatted key and slot (key first and slot second always!) This hash is where the value for a mapping is stored. A 32 byte hash can be converted to a number. This number will be somewhere in the range of 0 to ²²⁵⁶, a huge number as mentioned earlier. Due to the immense range, it is virtually impossible for a mapping value to collide with another value. Mappings are set this way because one cannot know ahead of time how many elements are going to be in a mapping. This way, it can be set in a non-sequential way where values will not be overridden by other storage.

In getMappingStorage, we use getMappingSlot and use the same web3.eth.getStorageAt() function we used before.

Testing for Mapping Storage

Let’s put the mapping storage helper we just created to use. Import getMappingStorage from helpers.js into StorageCoin.js and put this new test block in for storage slot 0 before the slot 1 test:

Here you can see we are checking that owner balance matches totalSupply. Remember that we give the owner the entire supply in the constructor of the contract. We can take a hex value and convert it to a number using BigNumber. We can then make sure that this matches totalSupply.

Run truffle test for yourself to see that all tests pass.

Understanding Nested Mapping Storage

In our StorageCoin contract we also have a nested mapping named allowed like such: mapping(address => mapping(address => uint256)). This essentially follows the same pattern as a regular mapping, but in a recursive fashion. Let’s hop into the helper function again to see how this works. We are going to add a new function called getNestedMappingStorage which will run the getMappingStorage function we already created recursively. Your helpers.js file should look similar to below with the new function:

As you can see, there is nothing to fancy about this function. It just runs the getMappingSlot function twice, taking a hash of the slot and key and then taking that hash and hashing it with the second key.

Let’s put the new function to use in a test in StorageCoin.js. We are going to need to set an allowance in order to see a value. In order to do this we are also going to need to unpause the token. Let’s write tests for both of these steps while we are at it. Here is the updated test file:

The test file has been updated in the following ways:

removed logs for simpleStorage at the start of the tests
added spender to top level of tests after owner
added test for unpausing
added test for setting allowance
added test for finding nested mapping storage

Run truffle test and check that the tests pass. You should get something similar to this:

when accessing StorageCoin storage
    Contract: StorageCoin
      ✓ should have owner balance in slot 0 mapping (88ms)
      ✓ should have totalSupply in slot 1
      ✓ should have paused in slot 3
      ✓ should have owner in slot 3 as well
      ✓ should have name in slot 4
      ✓ should have symbol in slot 5
      ✓ should unpause the token as owner (56ms)
      ✓ should set allowance for spender as owner (83ms)
nested mapping slot:
0xded101565a23504cd4339827add3a8b16f30ceff2912c1b6ef015848eade7942
nested mapping value slot:
0xd40f44dd33ce83ebf23375ff0b844b9aa265bbdfdf680d1743d07ed5bca013e5
nested mapping value storage:
0x015af1d78b58c40000
allowance as number:
25000000000000000000
      ✓ should have owner allowance for spender at correct storage slot (109ms)

You will also see the different values for our nested mapping. We already know through from earlier that your allowance slot is in slot 2, so that is our first mapping slot. We then take that value and hash it with owner and get the value printed for nested mapping slot. This is where our nested mapping exists much like our first mapping exists at slot 2. We then take this value and hash it with spender to get our nested mapping value slot. This is the actual storage location. Nested mapping value storage is the actual value which is in hexadecimal. Turn this into a decimal by putting it into BigNumber and we get the last value, the decimal value of the allowance owner has given to spender.

In the test itself we take the decimal value and compare it to the value we get from normal retrieval through the getter function allowance().

When done with this remove the console.logs if you feel like being tidy 😀.

Building Some Neat Tools for Mappings

We have found all of our storage… but this is a bit easier when someone is holding your hand. What about if you were looking at a completely different contract which was much more complex? It might get really difficult to actually know which slot is where the mapping should be.

Lets create a couple finder functions which will find mappings and nested mappings. Add these two functions to the bottom of your helpers.js file and remember to export them:

There isn’t a whole lot to explain for each of these functions. We are just iterating over slots from start slot to end slot and returning any values we find.

Let’s play with it in our test file. Add these two tests to the end of the test block:

As you can see we are given the same arguments for previous related functions. We are just setting it to check from slot 0 to 20 for each function. This can be quite useful for finding complicated storage.

Run yarn test to see the results and check that the tests pass.

Conclusion & Where to Go Next

There nested mappings are probably about has complicated as it gets when accessing storage. However, we have not covered arrays in this tutorial. If you want to check on how to retrieve additional types. I would recommend either checking the links at the top of this article, or checking the official solidity documentation on the matter. It is also worth repeating as it was only briefly mentioned: if two variables can both be fit into a 32 byte slot, they will both be packed into the same slot. We saw this with the boolean getting packed in with the address. Keep this in mind when looking through storage!

You should now have (most of) the tools you need to check through any contract’s storage, private or not.

Join Coinmonks Telegram Channel and Youtube Channel get daily Crypto News

Also, Read

Copy Trading | Crypto Tax Software
Grid Trading | Crypto Hardware Wallet
Crypto Telegram Signals | Crypto Trading Bot
Binance Trading Bots | OKEx Review | Atani Review
Best Crypto Trading Signals Telegram | MoonXBT Review
How to buy Shiba(SHIB) Coin on Bitbns? | Buy Floki
CoinFLEX Review | AEX Exchange Review | UPbit Review
10 Best Cryptocurrency Blogs | YouHodler Review
Best Crypto Exchange | Best Crypto Exchange in India
Best Crypto APIs for Developers
Best Crypto Lending Platform
Free Crypto Signals | Crypto Trading Bots
An ultimate guide to Leveraged Token