A Complete Mental Model for Ethereum dApp Development

Blockchain Application Development for Web Developers

Photo: https://www.ethereum.org/assets

Introduction

The prevailing architecture of most web applications is client-server-database. We utilize clients such as laptops, smartphones, and tablets to make requests from and submit requests to backend servers, which in turn communicate with databases to persist and retrieve data. The relationship between client and server/database is many-to-one, where control is centralized at the latter. If Web 1.0 is about static web pages, Web 2.0 is about interactivity and JavaScript.

web stack: Clients => Servers => Databases

Where we want to go is Web 3.0, where servers and databases are as decentralized as the clients. In other words, clients can also act as servers or databases or both (aka, peer-to-peer). With many-to-many relationships at all levels of the stack, there are no concentrations of control and a single point of failure. The network is maximally distributed.

Web 1.0 
Static (HTML/CSS)
Web 2.0
Interactive (JavaScript)
Web 3.0
Decentralized (Blockchain)

Why do we want clients to also maintain the databases? The most obvious and powerful use-case is Bitcoin. If we all collectively maintain a public ledger of balances and transactions, we can create a global currency that’s completely divorced from governments and institutions. With prophetic timing, “Satoshi Nakamoto” mined the first bitcoin on January 3, 2009 at the height of the worst financial crisis since the Great Depression.

blockchain => public ledger => global currency

Soon after, Vitalik Buterin realized that if clients could also maintain the “servers” along with the databases, the underlying blockchain can become programmable. The incredible implication is autonomous smart contracts. Such contracts could then be programmed to tokenize (digitize) real-world assets with records of ownership that are immutable. In effect, anyone from any corner of the globe can trade anything, cheaply and securely, without the need for intermediaries which are prone to corruption. From land, energy, music, and voting, the applications are endless, disruptive, and profound.

smart contract => records of ownership => blockchain

Applications that utilize smart contracts are known as decentralized applications, or dapps for short. Dapps are frontend apps that interface with smart contracts (instead of servers) to persist or retrieve data on the blockchain (instead of databases). Clients interact with dapps through external accounts. To help web developers looking to get into Ethereum dApp development, the author hopes to impart a simple mental model for these architectural layers that make up the blockchain application stack.

webapp: Client           => Server         => Database
dapp: External Account => Smart Contract => Blockchain

For a nontechnical introduction to blockchain, check out this insightful (and quite funny!) primer by Bertie Spell:

https://bertiespell.com/blockchain-solves-everything

Blockchain (Database) Layer

The blockchain layer is analogous to the database layer, where data is persisted and retrieved. Here, blocks record every transaction that has ever taken place. What makes blockchain different is that anyone can add to the chain. Consensus is achieved through a process called mining, where computers running an Ethereum client compete to secure the next block (aka, proof-of-work). Together, these nodes make up a network, and each one retains a copy of the entire blockchain. It’s a novel solution to the Byzantine Generals Problem that plagues all distributed systems.

network: Ethereum clients => mine(blocks) => proof

To add new data to the blockchain, we submit transactions to nodes in the network. Nodes then group pending transactions into blocks before beginning the mining process.

node: transactions => blocks => blockchain

To be precise, each block is a Merkle tree of transactions, plus a Merkle tree of receipts and a Merkle tree of the state. The benefit of these Merkle trees (Merkle-Patricia tries to be exact) is a simplified verification process (aka, Merkel proof), where nodes can simply download the block headers instead of every transaction in every block to verify a payment (aka, light clients).

Block Header:
1) transactions => transactions trie => transactionsRoot
2) infos and logs => receipts trie => receiptsRoot
3) addresses and balances => state trie => stateRoot

transactionsRoot is a trie of transactions in the block, receiptsRoot is a trie of transaction infos and logs, while stateRoot is a trie of account addresses and balances (aka, ledger). Essentially, a transaction is just a state transition, and a smart contract is just a state transition function.

transactions: genesis => state 1 => state 2 => ... => state N

And the underlying data structure is basically a linked list of binary trees of cryptographic hashes.

blockchain: genesis <= tries 1 <= tries 2 <= ... <= tries N

To mine a new block, we calculate a hash of these Merkle roots in the block header. This hash must reference the hash of the last block while it computes a special nounce that results in a value less than some target. This mathematical work is “proof” that the block is valid.

hash(nounce, block header, previous hash) < target

The target is set by the Ethash algorithm which takes into account the time it took to solve the previous block such that the average time normalizes to 15 seconds (aka, block time). This target is continually adjusted because the number of people running nodes is not always constant and the quality of computing resources keeps improving. The output hash is a hexademical (base 16, usually prefixed with “0x”) and is always 64 characters long.

https://etherscan.io/chart/blocktime

A block time that’s too high (low target value) will slow down the network and increase processing time of transactions. A block time that’s too low (high target value) will create many duplicate solutions, creating forks in the blockchain. A block time of 15 seconds seems to be ideal.

low target value => high block time => takes too long
high target value => low block time => too many forks

When forks occur, Ethereum follows the GHOST protocol which selects the fork that has done the most “work”, i.e., the longest chain (highest block number).

Fork A: genesis + block 1 + block 2 + ... + block 12 // main
Fork B: genesis + block 1 + block 2 + ... + block 11 // uncle
Fork C: genesis + block 1 + block 2 + ... + block 10 // uncle

Unlike Bitcoin, forks in Ethereum are not “orphans” and abandoned. They are “uncles” and are rewarded to create an economic incentive that decentralizes mining pools for increased security. Generally, we wait for at least 6 network confirmations to allow the longest chain to win by a wide margin.

block reward: 3 + fees + (1/32 x uncles)
uncle reward: 7/8 x 3 // 2 uncles per block (maximum)

The more mining is centralized, the greater the risk of a 51% attack, where miners collude to validate double spendings. Waiting for at least 6 more blocks to be confirmed by the network also decreases this risk. Either way, such attacks are impractical and unprofitable, costing at least $6 billion to carry out, and even more if we consider the opportunity cost of not acting honestly and receiving rewards in return.

Let’s take a look at an actual block of transactions on the Ethereum network:

https://etherscan.io/block/5912705

The block Height indicates its order in the chain. In this case, it is block number 5,912,705. According to the TimeStamp, this block was mined on July 5, 2018 around 12 AM by Ethermine. This block contains 89 transactions, which amount to 19,290 new bytes of data. Ethermine was the first to determine the correct Nounce that results in a Hash value less than the target Difficulty. As a reward, Ethermine received 3.46486964996 ethers (Block Reward).

block reward = coinbase + fees + uncles = 3 + 0.46486964996 + 0

Gas represents computational operations required to execute code. The block Gas Limit defines the maximum number of operations all the transactions in the block are allowed to “consume”. Like Bitcoin’s block size limitation, its purpose is to keep processing time of transactions and propagation time to other nodes low. Unlike Bitcoin, however, it’s not a constant. Miners have the option to adjust this limit by a small factor.

block gas limit = 1.5 x 10^6 x π ± 1/1024
sum(transaction gas limits) < block gas limit

As in Bitcoin, ethers are created whenever miners successfully solve the cryptographic puzzle. This is done simply by incrementing the miner’s account balance by the reward amount (aka, coinbase transaction). In Bitcoin, however, an actual coinbase transaction needs to be issued by the miner.

Let’s now take a look at two of those transactions:

https://etherscan.io/tx/0xc28ab33c5943d0d593d34d9af1b29971310dac009b88b83d3b2fae5dfdfed327
https://etherscan.io/tx/0x0875ed575d905efdbb909020da5f83867aadfa572f4164ff009a94025f4a977e

These two transactions represent the two possible types in Ethereum. We can send ethers to another account (first transaction) or we can send them to a contract account (second transaction).

1) external account => external account
2) external account => contract account
3) contract account => contract account // "internal" transactions

Both transactions tell us the TimeStamp, From, To, and how much ether to send (Value) . They also indicate how much gas was available (Gas Limit), how much was consumed (Gas Used By Txn), and its unit cost (Gas Price). The Nounce here is just a counter to give order to all the transactions being submitted by a particular sender. This is necessary because transactions need to be processed in order to prevent people from spending more than they have. Upon submission, the TxHash is created. It’s the ID of a transaction, which we can use to ping the network to check on the TxReceipt Status of our transaction. If our transaction is meant for a contract account, we can attach Input Data for the contract to consume.

Gas Used By Txn < Gas Limit
Gas Used By Txn x Gas Price = Actual Tx Cost/Fee

When we submit transactions, we’re asking nodes in the network to process them for us. This requires compute resources, which we pay for in the form of gas. Gas fees not only deter spam, but also halt bad code such as infinite loops. We can specify the amount as well as the price. At the start, the total potential cost will be deducted. Any gas left over will be reimbursed. If we set an insufficient amount, our transaction will revert and we will not be refunded. The more we’re willing to pay per unit of gas, the more priority we will receive. At present, a gas price of 50 gwei ($0.50) has a processing time of 1 minute. Compared to remittance transfers, this speed is incredible and the cost is minuscule.

gas price = supply (miners) + demand (external accounts)

Quite confusingly, contract accounts also have a nounce with yet another meaning. In these cases, this “nonsensical” number is incremented whenever the contract account creates another contract account.

block nounce = solution to the cryptographic puzzle
external account nounce = number of transactions issued
contract account nounce = number of contracts created

Before we conclude this section, let’s examine the makeup of external and contract accounts:

Photo: https://etherscan.io/address/0x68b42e44079d1d0a4a037e8c6ecd62c48967e69f
https://etherscan.io/address/0x06012c8cf97bead5deae237070f9587f8e7a266d

Both types of accounts can send and receive ethers. Therefore, both have a Balance field to keep track of them. Contract accounts, however, can also store data. Therefore, they have a storage field, and a code field that contains machine instructions on how to manipulate the stored data. In other words, contract accounts are smart contracts that live on the blockchain.

contract account = balance + code + storage
external account = balance + empty + empty

External accounts are controlled by private keys (i.e., humans) while contract accounts are controlled by code. A transaction between two external accounts is simply a value transfer. A transaction from an external account to a contract account activates the called account’s code. Contrary to popular belief, smart contracts cannot self-execute. Actions on contract accounts are always initiated by external accounts.

external account => transaction => contract account A => "internal" transaction => contract account B => ...

In Ethereum as in Bitcoin, we can only send ethers; we can never take. Therefore, the security of the From field is paramount. It’s so vital, in fact, that three additional fields are used to determine its validity. They are the v, r, and s. Obviously, if we can spoof any From address, we can manipulate any account to transfer all its ethers to us. To comprehend v, r, and s, we need to understand asymmetric cryptography.

transact = send ethers // can never take ethers

We use cryptography to encrypt and decrypt messages. With symmetric cryptography, the same key can be used for both encryption and decryption. With asymmetric cryptography, only the private key can be used for decryption, while anyone can use our public key to encrypt messages that only we can see.

// Symmetric Cryptography
encrypt(unencrypted message, key 1) => encrypted message
decrypt(encrypted message, key 1) => unencrypted message
// Asymmetric Cryptography
encrypt(unencrypted message, public key) => encrypted message
decrypt(encrypted message, private key) => unencrypted message

This property makes asymmetric cryptography more secure because we have no reason to share our private key with anyone. Another advantage of asymmetric cryptography is the ability to “sign” a message with our private key. Then using our public key, anyone can verify that our message and our “signature” match.

sign(message, private key) => signature
verify(message, signature, public key) => true/false

In Ethereum, private keys are random 64 hexadecimal characters. In base 10, they are incredibly large and impossible to guess. From private keys, we can generate public keys using the Elliptic Curve Digital Signature Algorithm (ECDSA) to get 128 hexadecimal characters. To generate account addresses, we hash public keys using Ethash (keccak-256) and drop the first 24 characters to arrive at 40 hexadecimal characters.

random(hexadecimal characters) => private key // 64 characters
ECDSA(private key) => public key // 128 characters
keccak(public key) => account address // 40 characters

In Ethereum, r and s are outputs of signing the transaction object with the sender’s private key using ECDSA, while v can be used to compute the originating address.

sign(transaction) => r and s (signature)
v => public key => account address

Together, these parameters are the means by which nodes in the network can verify that the transaction in fact originated from the account address in question.

verify(transaction, rs_signature, v_address) => true/false

Because it’s possible to derive the originating address from v, it’s not necessary to submit it along with the transaction!

Finally, contract addresses can then be generated from the account addresses and the transaction nounce:

keccak(account address, transaction nounce) => contract address // 40 characters

We ignore the first 12 characters to also arrive at 40 hexadecimal characters.

Because external addresses and contract addresses ultimately come from our private keys, as long as we keep them safe, theft is impossible. Conversely, if we ever lose our private keys, our ethers are lost forever.

Smart Contract (Server) Layer

The smart contract layer is analogous to the server layer, where the business/controller logic resides. To create a smart contract, we simply submit a transaction to a “blank” recipient with our logic as payload from an external account.

external account => "blank" transaction => contract account

On the blockchain, smart contracts exist as “internal” accounts with additional storage and code fields. Together, these fields persist the data and the logic of smart contracts, respectively. As such, smart contracts can be abstracted as class constructs, and a Turing-complete programming language can be invented. Solidity is such a language, and it’s the most popular because of its similarity to JavaScript.

storage + code = properties + methods => class => Solidity

In Solidity, the class construct has the keyword contract. Like JavaScript, variables and functions (aka, properties and methods) can be defined inside these contract classes.

pragma solidity ^0.4.24;
contract {
variables;
    constructor {}
    functions {}
}

Like classes, any contracts can inherit any other contracts:

pragma solidity ^0.4.24;
import "/Foo.sol";
contract Bar is Foo {
...
}

Here, Bar inherits from Foo, meaning Foo’s state and functions are transferred to Bar.

inheritance: base contract => derived contract
// inherits variables and functions, which can be overridden

Foo’s functions can be overridden if Bar’s functions have the same name, inputs, and outputs. If name is the same but inputs and outputs are different, the function will be overloaded instead.

function foo(uint a) {}
function foo(uint a) {} // overridden
function foo(uint a, uint b) {} // overloaded

If more than one contract is inherited, we must specify the order from the “most base” to the “most derived”.

contract A {}
contract B is A {}
contract C is A, B {} // will not compile!
contract C is B, A {} // will compile

If composition is preferred over inheritance, contracts can call library functions without inheriting:

pragma solidity ^0.4.24;
library Foo {
function foo() {
...
}
}
contract Bar {
function bar() {
Foo.foo();
}
}

Though a single contract can inherit from many others, on the blockchain, the codes are combined into one contract account. On the other hand, codes from libraries exist independently on the blockchain.

inheritance: multiple smart contracts => one contract account

Library accounts on the blockchain do not have storage and cannot hold ethers. When library functions are called, the calling contract’s storage and balance (aka, context) are passed in. Linking to the library contract is possible only after it has been deployed because its address on the blockchain is needed by the contracts that depend on it.

1. deploy(library) => library address
2. deploy(contract(library address)) => contract address

Solidity supports object-oriented design. We can create abstract contracts that others can inherit; just the function signatures can suffice.

pragma solidity ^0.4.24;
contract Foo {
function foo() public returns (bytes32);
}
contract Bar is Foo {
...
}

We can also create interface contracts. Here, only the function signatures are allowed. So, no constructor, no variables, and no inheritance.

pragma solidity ^0.4.24;
interface Foo {
function foo() public returns (bytes32);
}

Solidity files should have the same name as their contract classes, and the extension .sol. To compile them, we specify the version of Solidity we want to use with a pragma statement. The outputs of the compilation process are the bytecodes and the application binary interface (ABI).

pragma solidity ^0.4.24
semver => major.minor.patch
^ => the version indicated up to but not including the next major version
~ => the version indicated up to but not including the next minor version
compile(contract.sol) => bytecodes + ABI

To deploy smart contracts, we submit “blank” transactions with the bytecodes attached. Whenever a contract is deployed, an “instance” of it is created and its constructor method is called. Along with the bytecodes, we can also attach the arguments that we want to pass to the constructor method.

contract class => bytecodes + constructor arguments => contract instance

Analogous to the JavaScript Engine, the Ethereum Virtual Machine (EVM) resides in every node and is responsible for executing the bytecodes. As the EVM runs, it maintains a stack of opcodes that can write to the contract’s storage tree. This stack can have up to 1024 elements, and each element is 32 bytes. Thus, recursion is not recommended because there’s a good chance it might cause a stack overflow. In essence, the EVM is simply a state machine, and the opcodes specify how state transitions are applied to the next block state.

EVM stack = opcode + opcode + ... + opcode => next block state

Some opcodes are more expensive to execute than others:

Operation         Gas           Description
ADD/SUB           3             Arithmetic operation
MUL/DIV 5 Arithmetic operation
ADDMOD/MULMOD 8 Arithmetic operation
AND/OR/XOR 3 Bitwise logic operation
LT/GT/SLT/SGT/EQ 3 Comparison operation
POP 2 Stack operation
PUSH/DUP/SWAP 3 Stack operation
MLOAD/MSTORE 3 Memory operation
JUMP 8 Unconditional jump
JUMPI 10 Conditional jump
SLOAD 200 Storage operation
SSTORE 5,000/20,000 Storage operation
BALANCE 400 Get balance of an account
CREATE 32,000 Create a new account using CREATE
CALL 25,000 Create a new account using CALL

During a contract-creating transaction, the EVM initializes a new contract account by executing the constructor function. The code that results from this initial execution is the actual code that makes up a particular “instance”.

EVM(bytecodes, constructor arguments) => contract instance(s)

Of course, this means we can deploy as many “instances” of our smart contract “class” as we want.

The ABI is the developer’s portal to the opcodes in the contract instance. Its function is to translate the bytecodes into JSON RPC calls to the network.

developer => ABI => RPC(bytecodes) => contract instance

To interact with our instance on the blockchain, we can use the ABI to instruct the EVM which functions and with which arguments we want to call. The most popular JavaScript library for this is web3, so named to remind us that we’re building the future of Web 3.0!

web3(ABI) => EVM(functions, arguments)

Each web3 invocation is either a transaction (send) or a call. Transactions are sent to the network and potentially state-changing. Calls are read-only and fast. Contracts can call other contracts (aka, message), but it’s always a transaction that gets things started. In other words, message calls in themselves are never state-changing, but they can be part of a state-changing transaction.

transaction (send) => message call(s) => state change
call => no state change

With web3, we can submit four types of transactions:

  1. Send ethers from one external account to another external account (like a Bitcoin transaction)
  2. Send a “blank” transaction to deploy a smart contract (becomes a contract account)
  3. Send ethers from an external account to a contract account
  4. Send a transaction to execute a method within the contract account (to update or retrieve the contract state, or call other contracts)

Transactions and calls are executed by the EVM as opcodes, which equate to gas. Transactions that update the blockchain or contract state require payment of ethers. These will take time and will always return the transaction hash. Calls that simply retrieve data from the blockchain or contract are “instant” and free. We can return any values we want from these calls.

setterFunction(ethers) => transaction hash // takes time
getterFunction(free) => any data // happens "instantly"

Because nodes compete to mine the next block, function calls are executed redundantly across the network. To offset this waste of computing resources, a best practice is to perform as many calculations off-chain as possible.

off-chain work => on-chain work

Though syntactically similar to JavaScript, Solidity has several major differences. Firstly, variables are strongly typed and there are two types. State variables are primitives, while reference variables point to collections of state variables. If implicitly convertible, types can be coerced into one another.

State variables:

  • bool: boolean value (true or false) // default is false
logical operators:
!   // logical negation
&& // logical conjunction
|| // logical disjunction
== // equality
!= // ineqaulity
conditional statements follow short circuiting rules
  • int / uint: positive or negative integer (²⁸ to ²²⁵⁶) // default is 0
- can be defined with or without a number suffix
- suffix must be a multiple of 8
uint = uint256
uint8, uint16, uint64, ... , uint256
int = int256
int8, int16, int64, ... , int256
  • fixed / ufixed: positive or negative point number // default is 0
fixedMxN or ufixedMxN
// M = number of bits taken by the type; between 0 and 80
// N = how many decimal points are available; between 8 and 256; must be divisible by 8
ufixed = ufixed128x18
fixed = fixed128x18
  • address: 20 bytes for holding an Ethereum address (40 hexadecimal characters) // default is 0x
<address>.balance 
// get balance
<address>.transfer() 
// send ether from current contract account to <address>
// errors will throw
<address>.send() 
// low-level counterpart to transfer()
// not recommended because errors are silent (return false)

We can also access account variables and functions by calling built-in properties and methods within all addresses. To get the account balance, we can call balance. To send ether to an address from the calling contract account, we can use transfer() (preferable over using send()).

<address>.balance => balance in wei
<address>.transfer() => transaction hash

To interact with other contract accounts, we can use call and delegatecall (callcode is deprecated) to execute functions within them.

<address>.call() 
// call another contract
// return boolean
// additional modifiers: .gas() or .value()
<address>.callcode() 
// deprecated
<address>.delegatecall() 
// delegates a function call to another contract

Call executes “externally” while delegatecall executes “internally”. In other words, delegatecall executes the function of another contract as if it were its own. Equivalent to calling a library function, delegatecall is capable of loading code dynamically from a different contract at runtime. The calling contract’s context (i.e., address, balance, and storage) is preserved; only the code from the called contract is taken.

External:
<address>.call("foo", 1)
Internal:
<address>.delegatecall(bytes4(keccak256("bar(uint)")), 2)
<library>.bar(2)

Reference variables:

  • fixed array: array of single-type elements and unchanging length, (byte, type[N])
byte[N] // 0 < N < 33
- can be allocated to storage or memory
- storage arrays can be any data type
- memory arrays can be anything but a mapping
- declaring the array public creates a getter function that requires the index of the desired value as a parameter
  • dynamic array: array of single-type elements and changing length (string, bytes, type[])
bytes = byte[]
.length
// returns the length of the array
// dynamic arrays in storage can be resized by assigning a length
.push()
// appends a value to the array
// new length is returned
// storage arrays and bytes (not strings)
uint[] memory a = new uint[](<variable length>)
// variable length can be defined at runtime by using the new keyword
// once defined, it will be of fixed size
  • mapping: collection of key-value pairs of same type
mapping (<key type> => <value type>)
// key type can be anything but mapping, dynamic array, contract, enum, or struct
// value type can be anything
// mapping is basically a hash table
// every value is initialized to its default
// no length property
// key data is not stored, only its keccak256 hash
// declaring the mapping public creates a getter function that requires the key of the desired value as a parameter
  • struct: collection of key-value pairs of different types
- a way to define new types
- cannot contain a member of its own type
- struct values stored as location variables are passed by reference
  • enum: finite set of custom types
- a user defined type
- explicitly convertible to integers
- requires at least one member

Like a computer’s hard drive and RAM, storage variables point to persisted states while memory variables point to temporary objects, respectively.

storage variable => persisted across executions // expensive
memory variable => persisted during execution // cheap

Obviously, state variables are always in storage while local variables of reference type are there by default and can be copied to memory by using the memory keyword. In other words, they are passed by reference. On the other hand, function parameters and return parameters are in memory by default. They are passed by value. To pass them by reference, we can use the storage keyword. Local variables of value type are stored in the EVM stack until the execution completes.

storage: state variables (always), local variables of reference type (default)
memory: function parameters (default), return parameters (default)
call stack: local variables of value type (always) // cheapest

Similar to memory is an area called calldata where parameters of external functions are stored.

calldata: external function parameters

Though reference variables obviously reference variables in storage, for clarity, we should specify the storage keyword anyway. To copy the referenced variables to memory, we can use the memory keyword.

mapping (address => uint) balances;
uint storage balance = balances[msg.sender]; 
// points to sender's balance in storage
uint memory balance = balances[msg.sender]; 
// copies sender's balance to memory

Bugs can hide in assumptions about where variables are being stored. This practice prevents this.

Like JavaScript, Solidity functions are first-class constructs, meaning they can be passed as parameters to other functions, or be returned by them. Therefore, there are two types of functions. Internal functions are called internally by other functions within the same context (aka, inter-contract message calls). External functions are called externally via transactions (from an external account or another contract). By default, functions are internal but can be changed by using the external keyword.

External: EVM => contract A => function 1
Internal: contract A (function 1 => function 2)

Because contracts can inherit from other contracts, we can also explicitly specify the accessibility of our functions as either public or private. Public functions can be accessed externally via transactions or internally by derived contracts, while private functions can only be accessed by the current contract. If missing the accessibility, functions are assumed to be public.

Public: contract A (function 1) => contract B (function 1)
Private: contract A (function 1) => contract A (function 2)

Though the public and external designations both enable functions to be accessed from the outside, a best practice is to favor external if we expect the functions to ever be called externally. This is because in public functions, the EVM copies the arguments into memory, while external functions just read the arguments stored in the payload. This copying step is expensive, especially for large arguments.

public functions: write arguments to memory // expensive
external functions: read arguments in payload // cheap

Type and accessibility of variables can also be specified. As “internal” properties of contracts, state variables are always internal. Like functions, they can be either public or private depending on whether we want derived contracts to access them. For convenience, automatic getter functions are created for public variables.

public variable => variable()

It’s important to note that although the private designation prevents other contracts from accessing and modifying its data, the nature of the blockchain is such that everything inside a contract is visible to all external observers. In fact, nothing is ever “deleted” from the blockchain. When we make a change to our contract state, it does not “overwrite” anything. Though it’s possible to use the selfdestruct operation to “remove” the data in the storage and code area of our contract, nodes have the option to keep them indefinitely.

selfdestruct(recipient) => contract "deleted"

Functions also come with default modifiers. For clarity, functions that do not modify the contract state should be marked as either pure or view. View functions need to read from the state while pure functions do not. Functions that modify state will always return a transaction hash, and nothing else. Functions that handle ethers should be designated as payable. Functions can also be decorated with custom modifiers to easily add additional behaviors and promote code reuse:

modifier onlyOwner() {
require(msg.sender == owner);
_;
}
function foo() onlyOwner {
...
}

Underscore indicates when the function body of the decorated function should execute.

In summary, functions have the following signature:

function doSomething(<parameter types>) {external|internal} {public|private} [pure|view|payable] [modifiers] [returns (<return types>)]
<> = required
{} = recommended
[] = optional
signature = name + parameter types + function types + accessibility + default modifiers + custom modifiers + return types
// explicit function signatures allow for easy translation to ABI

To summarize the function types and accessibilities in order of least restrictions:

public: accessible by all
external: only accessible from outside
internal: accessible by derived contracts
private: only accessible from within

Function parameters are defined by type and name. Name cannot be a reserved keyword.

function foo(uint a, uint b) {}

Functions can return multiple values. If so, they must be declared in the function signature. There are two ways to define the return parameters:

Type with name:
function foo() returns (uint a) {
a = 3; // return statement not necessary
}
Type without name:
function foo() returns (uint) {
return 3; // return statement required
}

Contracts can have one anonymous fallback function. This function cannot have arguments and cannot return anything, but a payload can still be supplied with the call. There is a gas limit of 2300, so the function should not do much. The fallback is executed whenever calls to the contract specify functions that do not exist or the payload is empty (i.e., a plain ether transfer). In these cases, for the contract to be able to receive plain ethers, the fallback function must be marked with payable.

pragma solidity ^0.4.24;
contract Token {
    function () payable {

}
}

Though web3 always ultimately returns the transaction hash for calls that update state, state-changing functions can still return values for other contracts to consume (aka, inter-contract message calls). Nevertheless, it’s still possible for such functions to emit events which web3 can watch for using the transaction hash.

Like JavaScript’s console function, events can be used for logging. When called, it causes its arguments to be stored in a special data structure on the blockchain (aka, transaction log). We can emit events within any function, and then watch for them using callbacks via the ABI.

Example event:

pragma solidity ^0.4.24;
contract Token {
event Transfer(
address indexed _from,
address indexed _to,
uint256 _value
);
function transfer(address _to, uint256 _value) public {
...
emit Transfer(msg.sender, _to, _value);
}
}

Example callback:

const token = new web3.eth.Contract(ABI, ADDRESS);
const event = token.Transfer();
event.watch((error, result) => {
if (!error) console.log(result);
});

Note: Up to 3 event parameters can be indexed for faster queries.

To summarize, contract declarations can contain:

1. state variables
2. reference variables: arrays, structs, enums, mappings
3. events
4. function modifiers
5. constructor function
6. fallback function
7. external functions
8. public functions
9. internal functions
10. private functions
// ideally in this order for convention
// within each function group, state-changing functions should come before read-only functions (view before pure)
// variable and function names should be camelcase

Like JavaScript, there are special variables and functions that always exist in the EVM’s global namespace. To access information about the transaction or the block, we can use the msg and the block global variables, respectively.

Transaction properties:

  • msg.data (bytes): the payload
  • msg.gas (uint): remaining gas
  • msg.sender (address): sender of the message call
  • msg.sig (bytes4): function identifier (first four bytes of calldata)
  • msg.value (uint): number of wei sent with the message call
  • tx.gasprice (uint): gas price of the transaction
  • tx.origin (address): sender of the transaction // could be insecure
  • gasleft()(uint256): remaining gas

Block properties:

  • block.blockhash(uint blockNumber) returns (bytes32): hash of a given block (no later than 256 blocks ago)
  • block.coinbase (address): address of block miner
  • block.difficulty (uint): target value for mining
  • block.gaslimit (uint): total gas limit for transactions
  • block.number (uint): height of block in the chain
  • block.timestamp (uint): time of mining (seconds since unix epoch)
  • now (uint): equivalent to block.timestamp

Utility functions include cryptographic helpers such as sha256 and keccak256. Time suffixes are also available such as seconds, minutes, hours, days, weeks and years.

now = block.timestamp
// miners can alter to some degree as long as timestamp is greater than previous block and less than the next block
// i.e., 30 second window
1 = 1 seconds
1 minutes = 60 seconds
1 hours = 60 minutes
1 days = 24 hours
1 weeks = 7 days
1 years = 365 days
// leap seconds are ignored

Ether units are also available: wei, szabo, finney and ether

1 ether = 10^18 wei
1 ether = 10^6 szabo
1 ether = 10^3 finney

Like dollars and cents, wei is the lowest denomination of ethers. It’s not possible to have fractions of wei.

1 dollar = 100 cents
1 ether = 1,000,000,000,000,000,000 weis
// on the blockchain, ethers are stored in domination of wei

We can throw exceptions to handle errors. A thrown exception will always revert the state.

  • assert: checks expected against actual (internal errors)
  • require: ensures valid conditions (external/input errors)
  • revert: always throws an exception

Like JavaScript, the this variable is also available for easy access to the contract instance itself (can be coerced to the contract’s address). Using this, it’s possible to call internal functions “externally” from within a contract.

foo() => internal call from within contract
this.foo() => external call from within contract via EVM
address(this) => returns contract address
this.balance // deprecated
address(this).balance // recommended

Inline assembly (EVM opcode) is also available to offer granular control over the EVM stack. Though beyond the scope of the typical Solidity developer, awareness of these low-level operations and limitations will prove invaluable during the design process.

Writing code is easy. Designing good code is not. For example, because of underlying gas costs and a gas limit on every block, we should not iterate through dynamic arrays of indeterminate size. Instead, we should design our data models in such a way that loops are unnecessary.

Poor Design:
address[] owners;
function isOwner(address owner) returns (bool) {
for (uint i; i < owners.length; i++) {
if (owners[i] == owner) return true;
}
return false;
}
Good Design:
mapping (address => bool) owners;
function isOwner(address owner) returns (bool) {
return owners[owner];
}

To summarize actions that read the state:

- reading from state variables
- reading from <address>.balance or this.balance
- reading from members of block, tx, or msg
// except msg.sig and msg.data
- calling any function not marked pure
- using inline assembly with "read" opcodes

To summary actions that modify the state:

- sending ethers
- creating contracts
- changing state variables
- emitting events
- calling any function not marked as pure or view
- calling selfdestruct
- using low-level calls
- using inline assembly with "write" opcodes

A quick and easy way to start writing and testing smart contracts is to use Remix, an online editor maintained by the Ethereum Foundation.

https://remix.ethereum.org

With the Remix editor, we can easily author smart contracts using the Solidity programming language. Remix can automatically compile our contract to help audit our code in realtime! When we’re ready, we can tell Remix to use “Injected Web3” (via Metamask) to deploy our contract to a real network.

It’s also possible for Remix to create a “fake” in-browser network to help us quickly test our contract! To tell Remix to host a local test network (aka, private network), select “JavaScript VM” for our Environment under the Run tab. For our convenience, five external accounts are automatically created and pre-funded with 100 ethers each.

To deploy our contract, we simply provide the arguments that we want to pass to our constructor function before clicking “Deploy”.

An “instance” of our contract now exists on our local network. To help us test our contract, Remix automatically creates input fields for every function that exists in our contract.

On the bottom panel, Remix provides a console that outputs all the things that are happening on our local network as we interact with our contract.

Note that transactions are processed instantly to save us time when in reality they take much longer. In this author’s opinion, Remix is the best IDE to use during the design phase of the development process!

External Account (Client) Layer

The external account layer is analogous to the client layer. Architecturally, external accounts exist independently from the network. This is desirable because anyone can create an Ethereum network, and many have been created:

- main Ethereum network
- remote test networks (Rinkeby, Kovan, Ropsten)
- remote private networks
- local test network
- network forks

With a tool like geth, we can become a node for the main network or our own. In essence, Ethereum is just a protocol, and anyone can implement it.

Ethereum protocol => geth => Ethereum network(s)

Because external accounts are completely decoupled, the same account can be used across networks! Hence, the name “external” account.

external account 1 => network A
external account 1 => network B
external account 1 => ...

Thus, external accounts are like “email addresses”. Just one can be used for authentication across all websites.

username + password => email address => any websites
public key + private key => account address => any networks

Just as an email address is created from a username and password, an external account address is created from a public key and private key. One is secret while the other is not. Like websites, dapps provide the user interfaces to the smart contracts that exist on the blockchain. Clients such as Chrome and Mist (or any web3-enabled browser) are the means by which users interact with websites and dapps.

User => Email Address   => Website => Server         => Database
User => Account Address => Dapp    => Smart Contract => Blockchain

Like having multiple email accounts, having multiple external accounts is useful for protecting our privacy on the public blockchain. However, managing all the various credentials can be frustrating. To ease this burden, we can use BIP39 to generate account addresses and key pairs, algorithmically from a “parent key”, ad infinitum. All we need to remember is a 12-word mnemonic phrase.

BIP39(12 words) => array[account addresses, public keys, private keys]

Backed by ConsenSys, Metamask is the most popular BIP39 account management tool. Just as web3 is the developer’s portal into the Ethereum blockchain, Metamask is the consumer’s portal. Internally, Metamask also uses web3 to programmatically connect to the network!

To create a vault, Metamask generates a random seed phrase (i.e., mnemonic phrase). To help us quickly access our vault without needing to input our seed phrase every time, Metamask asks us to create a password. With such a password, we can lock away our seed phrase in cold storage for maximum security.

Internally, each account has a public key, a private key, an address and a balance. For our privacy and security, our account credentials are encrypted and stored locally. Nothing is sent to Metamask. To send ethers, Metamask signs our transaction request using our private key before submitting it to our network of choice. To receive ethers, we simply provide the senders with our address so that they can initiate the request from their end.

With Metamask, we can select the main network as well as test networks. We can also setup a local network on port 8545, or provide a URL to a remote network.

To better understand the flow, let’s send some ethers using a test network so we don’t have to spend real money. Rinkeby is the most reliable because it uses proof-of-authority instead of proof-of-work. Because ethers in test networks have no real value, there are not enough miners incentivized by proof-of-work to create a healthy level of decentralization. Consequently, attackers can easily take over the mining power. With proof-of-authority, we don’t have to worry about such attacks because blocks are signed by trusted community members (aka, consortium).

proof-of-authority => trusted consortium
proof-of-work => competition based on mining power

In proof-of-work, hash rate (hashes per second) determines how likely a computer is to add the next block of transactions. To disincentivize the need for specialist hardware, Ethereum plans to move from proof-of-work to proof-of-stake. In proof-of-stake, anyone with ethers to stake can participate.

proof-of-stake => competition based on putting ethers at stake

Before we can send ethers, we need some first! To get free ethers, Rinkeby has a faucet we can use. The only requirement (to prevent abuse) is that we authenticate with our social profile.

https://faucet.rinkeby.io

Once we submit our account address, we should see that our balance has increased. Now that we have ethers to spend, let’s try to send some to another account in our vault. First, let’s create a second account.

Copy the address of the second account, then switch back to the first account and paste in the address. Provide an amount in ether to send. We can also attach a hexadecimal string as payload. Click Next.

Metamask will then ask us to confirm the transaction. Note, Metamask automatically calculates the appropriate gas limit and gas price for us!

To determine the best gas values for us, Metamask simulates the function call to our contract (aka, dry run). If there are errors, Metamask will display them too!

Click Submit. Metamask will now sign the transaction with our private key and submit it to the Rinkeby network.

For our convenience, Metamask records all of our outgoing transactions. Clicking any one will take us to the blockchain where it resides.

https://rinkeby.etherscan.io/tx/0x57cbaba8db5b1a868835356c05d1de661a811a7dd81f73a89b787b38665eed43

If we switch to our second account, we should see that our balance has updated.

Because external accounts are completely decoupled from the blockchain, the dapps we build will never ask for the user’s keys. We can package together the transaction object based on our user’s inputs, but ultimately, the user needs to sign it in order to validate it. Metamask is awesome because when we make a web3 call to submit a transaction, Metamask will bring up a modal to assist the user in the signing process. Therefore, we should build our dapps with Metamask and with this flow in mind.

web3(transaction) => Metamask popup => sign(transaction) => network

As we’ve seen, every transaction will cost some amount of money. Even a tiny update to the blockchain is not free. This means our users will have to pay every time they want to change something. Thus, our value proposition should outweigh these inherent costs.

value proposition > transaction fees

Conclusion

Traditional application stacks are clients, servers, and databases. In contrast, Ethereum application stacks are external accounts, smart contracts, and the blockchain. Like JavaScript and the browser, the EVM and the Ethereum client aim to be on every laptop and computer in the world. When the “server” and the “database” are also managed by the “client”, we achieve maximal decentralization and distribution. The effect, there are no single points of failure and concentrations of power. Information, value, and wealth are normalized and democratized. Such a network embodies our highest ideals of democracy and freedom.

Without the concept of ownership, a free market society is impossible. Trade cannot take place without clear titles of ownership at the outset. The disparity between the rich and the poor is ever increasing, but before we can reverse the trend, we first need to give everyone a means by which to definitively prove ownership. This has been unattainable until now because records of ownership are monopolized by banks and governments, and not all are honest and trustworthy. For a more equitable world to be possible, we need a “trustless” system that’s immune to corruption.

Ethereum is the beginning of such a system. Like a perfect world government, everyone is an equal citizen no matter the background. The only requirement is a private key that no one else has to know, even the “government”. Under such governance, smart contracts are like “banks”, each storing digital tokens of a real world asset. With a private key, we can permissionlessly enter any “bank” and buy anything from anyone, paying with one universal currency. The anonymous “signatures” we leave behind prove that the transaction took place, immutably and transparently. Ownership is assured. Privacy is protected. Trust and third-parties are disintermediated. Our world government and its banks are secured by cryptoeconomics, a combination of economic incentives and cryptographic verification.

A market society that’s truly free is paradisal because it tends towards perfect competition. When actors are able to compete with truthful information and legal guarantee, barriers to entry are reduced for new players. Eventually, a Pareto-optimum equilibrium will be reached such that nobody can be made better off without making someone else worse off. In other words, we undo the widening gap of economic inequality.

Trust => Freedom => Love => Peace

At HeartBank®, we believe the trustworthiest financial system is a trustless one. Human nature is paradoxical because we’re capable of both extremes. In the absence of trust, our defensive instincts can lead us to the most unimaginable destructions. Where we can trust and feel trusted, we have the capacity to rise up to meet the highest expectations. When we’re free to act from a place of peace, we naturally act with our ❤️.

The way to peace is trust(less).

You’re now ready to build a real Ethereum dApp! 😇 To learn how, check out:

References: