Getting Up to Speed on Ethereum

Go from zero to 💯 without all the nonsense.

Note: this article was written 1 year ago. It is very much still accurate and worth reading, and I encourage that, but be aware that the landscape has changed (it’s been 10 years in crypto-time 😂).

You should read this blog post if:

  1. You’re a professional software engineer
  2. You want to have a deep working understanding of Ethereum and the related ecosystem.

Prerequisites:

  1. You understand the concept of a blockchain and how Bitcoin uses it to create a trustless digital currency. No? Watch this short video or watch this long video. Hint, you should probably watch the long one.
  2. You’re a professional software engineer. Seriously, I mean, nontrivial amounts of real-world software engineering experience. Understanding systems/architecture/math is a learning efficiency multiplier.
  3. Like a week of your free time. I told you we’re skipping the nonsense.
  4. You understand the basic concept of a merkle tree and how it can be used to quickly verify information correctness.

Note: you don’t need to read the whitepapers, but you need to understand them. But it turns out that reading them is the fastest way to understand them. 😉

I recommend reading the entirety of this post, absorbing the contents at a high level, and then diving into each link individually, over the course of a few days.

You’ll be surprised at how little of this technology is magic (read: none of it), despite the severe case of buzzword-itis the ecosystem has. Everyone is building off of the shoulders of giants; if you make a living as a professional software engineer you have the ability to understand all of these projects and technologies at a deep level. Just don’t get distracted by the bullshit.

Ethereum

Obviously an acceptable explanation won’t fit into a paragraph. Go ahead and read the Ethereum whitepaper. Or one of the other trillion “how do I Ethereum/Blockchain/Smart Contract” posts on the internet. Or watch this video titled “Ethereum in 25 Minutes”.

Smart Contract

In Ethereum, contracts are given an address to uniquely identify them (it’s a hash of the creator’s address and how many transactions they’ve sent before). Clients can then interface with this address by sending it ether, calling functions, querying the distributed state that it manages, etc.

Smart Contracts, again, are just some code with some distributed state managed by a blockchain. For example, multi-sid wallets that you’re using to receive/send ETH is just a smart contract with a fancy UI on it.

This concept is pretty powerful, and I’m sure you’ve already read all about it. If you haven’t, browse your favorite mildly-technical new source (hey Medium!) and you’ll be inundated with people telling you how much potential there is. Some buzzwords: asset/rights management, decentralized autonomous organizations (DAOs), identity, social networking, etc.

Gas

The “price” of gas (i.e. “how much $$$ does adding two numbers together on this distributed computer cost me”) is determined by the market, similar to Bitcoin’s transaction fees. If you pay a higher gas price, nodes will prioritize your transactions for that sweet sweet profit.

In general, it’s way more expensive to compute and store things on Ethereum than it would be to do it in a traditional environment, but Ethereum gives your code all those nice properties we discussed above, which can be just as valuable.

In general, reading state is free and writing state costs gas. Here’s a more in-depth overview of the concept of gas:

Distributed App (dApp)

A Distributed App doesn’t need to store all of its state and perform all of its computation on the blockchain, which would be pretty expensive for end-users, but a distributed app does eventually store trusted state on the Ethereum blockchain, which anyone can then read. Many distributed apps also use technologies like IPFS and Golem (discussed later) to handle computation and storage off of the Ethereum blockchain, but in an equally decentralized manner.

The ethereum organization on github has a dapp-bin repository that has some references and examples. Make sure you check the activity on the file you’re looking at, though, cause this kind of information gets stale very quickly.

Also, we’ve finally agreed on the proper capitalization of “dApp” via twitter poll (so you know it’s statistically significant), so let’s all just write it as “dApp” then, yeah?

dApp Client

A dApp Client is literally just any “client” or “frontend” as you’d normally use the term in programming, except this client interfaces with the Ethereum blockchain (perhaps in addition to other services) somehow. These clients are frequently written in JavaScript because we haven’t yet finished converting everything in the world to NodeJS.

More seriously, most dApp clients are written primarily in JavaScript because it can be run in a web browser and everyone’s already got one of those. They’re also frequently written in Go due to a superior state of existing tooling available. It’s a viciously positive cycle of improvement, which means unless you really know what you’re doing, you get to choose between JavaScript and Go (and to a certain extent, Rust) for interfacing with the Ethereum blockchain and the protocols being developed on top of it.

* So there’s a little bit of confusion/discussion around the exact terminology/definition of “Distributed App”: is it just the smart contract(s)? Is it the entire backend of a system which, at some point, interfaces with the Ethereum platform to store trust? Or maybe it includes the client code, too, and the user interface so the whole bundle of stuff is called a “dApp”?

* I’ve gone ahead and defined as “the backend of a system that interfaces with the Ethereum blockchain”. This is different enough from “Smart Contract” to warrant its own concept and also implies (correctly) that anyone can create a client to interface with a distributed app

dApp Browsers

The primary purposes of a dApp browser are to

  1. Provide a connection to an Ethereum node (either to one hosted locally or remotely) and an easy way to change that connection to point to a different node (which might be connected to a different network),
  2. And provide an account interface (“wallet”) for the user so that they can easily interface with these dApps.

Mist is the official Ethereum dApp browser. It’s really just a pretty web UI for interfacing with an Ethereum node and sending transactions to/from Smart Contracts (dApps!).

Status is a mobile browser with a unique approach to the UX of using dApps.

Toshi is Coinbase’s foray into building an Ethereum wallet and browser. It bets big on the “wechat” + “chatbot” vibes.

MetaMask is a Chrome Extension that turns Chrome into a “dApps Browser”. Its core feature is that it injects web3, a JavaScript Ethereum client library, into every page, allowing dApps to connect to MetaMask’s hosted Ethereum nodes. The Chrome extension allows you to manage wallets and connect to the different Ethereum networks available.

Parity is a an Ethereum client (as well as a full-node implementation) that integrates with your web browser, turning it into a dApp browser.

Ethereum Nodes

Your node needs to know which blockchain to download and which peers to talk to; see below for a discussion of the different networks available.

You should probably go ahead and run all of these node clients with docker and some sort of persistent storage. If you don’t feel like running a node on your own, you can use a third party like Infura. There is also a way to run a local node for testing and development, discussed later.

If you’re distributing a dApp client to users, you don’t necessarily need to provide access to an Ethereum node as well; dApp Browsers provide the connection to any client that needs it.

Ethereum Tokens

Yeah, all of the “tokens” that you hear about are just numbers in a distributed hash table with an API (aka protocol) to add and subtract. Here’s what a basic token contract looks like (that’s 38 lines of code, half of it comments and whitespace).

Go ahead and read the ethereum.org tutorial on creating a crowdsale; you’ll see that it’s just a contract (Crowdsale) that interfaces with another contract (MyToken) which is just like the basic token contract linked above. It’s not magic.

People are using tokens for a variety of uses, and you’ll quickly see that imagination has no limits. Tokens are frequently used to incentivize interaction with a protocol, prove ownership of assets, proof of voting rights, etc. Fred Ehrsam from Coinbase has a good talk about tokens, why they exist, and how they’re being used.

Vitalik recently wrote a blog post analyzing token sale models, which is a good read.

ERC20 / ERC223 / ERC777 / ERC827 / All the Tokens

Due to some issues with the ERC20 spec (namely that it was made quickly, has small security concerns, and requires two transactions for payments), now we have more proposed token standards.

* there had been some confusion around ERC223 vs ERC23, but they are the same concept; the ERC number is 223, so this standard should be referred to as ERC223.

ERC721 / NFT Tokens

* there was also ERC821 but its ideas have been merged via community consensus into ERC721

Interfacing with Smart Contracts

If you’d like to programmatically interface with contracts, there are various Ethereum client implementations. For JavaScript, web3.js, ethjs, and ethers.js are popular. For golang, the abigen executable in go-ethereum provides go packages for interfacing with contracts. At the end of the day, though, it’s just a standard JSON RPC API, so you can always write your own adaptor for the language of your choice if once isn’t available. Some client libraries provide convenience functions as well, beyond simple function execution.

Run a local ethereum node for testing and development with Ganache (previously called ethereumjs-testrpc).

When you “deploy” a smart contract, all you’re really doing is sending a transaction to the 0-address ( 0x0 ) with the contract bytecode as an argument.

Here’s more information about transactions:

Truffle, Embark, Populous, Perigord & Others

Frameworks like Truffle, Embark, Populous, and Perigord standardize and automate a lot of the minutiae. They provide a nice developer experience for writing contracts, deploying contracts, and incredibly importantly, testing contracts.

Truffle (written in Node) has the most developer adoption and seems to have the most active development; follow the Getting Started guide to get up to speed.

This post has a lot of good information and uses truffle to deploy and interface with a contract.

Embark (Node) has similar but different ideas for how developers should structure projects.

Perigord (Go) is very similar to Truffle.

Populous (Python) is an actively developed python framework that satisfies the same niche.

When you first start playing around with contracts, you should avoid using a framework until you understand the value it provides, much in the same way you shouldn’t start learning how to write HTML with rails new . The easiest thing to do at first is use Remix to play around with the language and ideas.

ETHPM & NPM

Of note, you can also use NPM to provide smart contract code for distribution, which a lot of people do, notably Open Zeppelin.

Read the spec for more information and background.

State of the Networks

Ropsten — The primary Ethereum testnet using Proof of Work. This network, because of low computation volume, is easy to DDOS, split, and otherwise mess with. It was recently revived and is usable again after being temporarily abandoned after a spam attack. It consistently has a very low gas limit, for no good reason, so many contracts will fail to deploy on this network.

Kovan — A parity-client only testnet using Proof of Authority which advertises immunity to spam attacks and a consistent 4 second block time.

Rinkeby — A geth-client only testnet using Clique Consensus, which is therefore more resilient to malicious actors, despite the low computation volume. (my personal favorite)

You can also run your own private Ethereum network. The go-ethereum team built puppeth to configure a full network, complete with custom bootnodes, genesis block, and consensus rules, which is what powers the Rinkeby network. You can also run your own infrastructure, perhaps using kubernetes or docker-compose. But you probably won’t need to run a private network any time soon.

“Accounts” vs “Wallet”

A wallet is one of two things: it can be 1) a fancy interface for creating and sending transactions using your account (i.e. MyEtherWallet) or 2) just a smart contract (which, again, is just some code) that sends and receives ether; it’s not a native concept. Here’s the Solidity implementation of a wallet on GitHub that’s used by the Mist browser. They can come in many flavors like multisignature, paper, etc.

Now that we’ve correctly defined the two, prepare to see people confusing the two terms everywhere and labelling anything that sends/receives Ether as a Wallet and calling anything and everything an account.

EVM and the State of Smart Contract Creation

Solidity

Play around with Solidity in Remix, a web-based playground for Solidity contracts.

Here’s what Solidity looks like:

pragma solidity ^0.4.11;contract BasicToken {mapping(address => uint256) balances;function transfer(address _to, uint256 _value) returns () {
balances[msg.sender] = balances[msg.sender] - _value;
balances[_to] = balances[_to] + _value;
}
function balanceOf(address _owner) constant returns (uint256 balance) {
return balances[_owner];
}
}

LLL

Here’s an example ERC20 contract in LLL.

LLL looks something like:

(seq
(def 'node-bytes 0x00)
(def 'owner 0x20) ; address
(def 'set-node-owner 0x5b0fc9c3) ; setOwner(bytes32,address)
(def 'get-owner (node)
(sload (+ node owner)))
.... this is just for illustration purposes, this definitely won't compile.

If you’re learning, you probably don’t want to write anything in LLL.

Serpent

Vyper (previously Viper)

Bamboo

Others

Smart Contract Decompilation/Disassembly

Smart Contract Security

Because so many engineers developing for Ethereum and other smart contract platforms are coming from web development, this concept might be new and crazy.

There are a variety of different security-related traps your code could fall into, both at a language level and from a high-level logic perspective. When you deploy a production smart contract that handles real money, you need to be 100% confident in the operation of the contract.

ConsenSys has a beautiful repository of smart contract best practices that you should understand at a deep level.

Before you deploy a smart contract that’ll handle real cash money, you should probably open a bug bounty and have it pentested. If you’re handling RealMoney™, you should have your code professionally audited.

* there’s a selfdestruct() function that a contract can use to remove itself from the network.

Whisper

It uses the protocol shh which is pretty cute. There’s surprisingly little documentation and adopting of this protocol.

Although it hasn’t been updated in a while, here’s an example dapp using Whisper to implement a chat client.

Whether or not whisper is under active development is up for discussion.

Decentralized Autonomous Organizations (DAOs)

A side effect of this is that decision making, governance, and what color the houses should be painted is immutably stored on the blockchain (in the state of those contracts). Cool stuff.

Ethereum Classic and the DAO Hack

Aragon

Check out the aragon-core contracts for a better understanding of how it operates.

Swarm

Projects

IPFS & FileCoin

While this is a new protocol, there is an http gateway and a filesystem adaptor, meaning you can fetch IPFS content via http and mount the entire world wide filesystem on your local disk at /ipfs . IPFS also provides a naming system called IPNS (InterPlanetary Name Space), which allows for mutable state (recall that everything in IPFS is immutable). You can even use DNS TXT records to direct your IPNS client, allowing you to generate human-friendly links to data.

FileCoin is Protocol Labs’ effort to create a distributed market for storage on IPFS; aka, an incentive layer for providing storage to the network. The FileCoin consensus protocol (mostly) does away with wasteful Proof of Work and uses Proof of Replication and Proof of SpaceTime (yes, really) to ensure that a piece of data is replicated a certain number of times and is stored for a specific amount of time.

You should read the IPFS whitepaper, the FileCoin whitepaper, and the IPLD spec.

While FileCoin isn’t yet released, you can use the existing IPFS storage network to host the html/css/js of your dApp client as well as use it as a database using something like orbit-db.

There’s also some public backlash about how FileCoin is being implemented and distributed:

Augur

The contracts on GitHub for the Augur markets are a great read.

Gnosis

Golem

Skip the marketing talk and read the FAQ on GitHub for a better understanding.

0xProject

They’ve started by implementing 0x Portal (previously 0x OTC), a dApp that uses their protocol to transfer tokens directly between users. You can check out their contracts on GitHub. They launched the contracts to the mainnet and are working with the community to build out Relayers.

Skip the buzzwords and read the 0xProject FAQ.

Swap

Bancor

Bancor is a protocol (and a set of smart contracts implementing that protocol) that lets you create a token that 1) prices itself based on orders and 2) provides instant liquidity by holding another token (like Ether or any ERC20) in reserve as collateral.

Hey look here’s a whitepaper that explains it very well.

Stablecoins, MakerDAO, & Dai

The Maker DAO is a DAO that is managing the Dai stablecoin. Dai was recently released and is currently stable; managing to keep its value during two notably turbulent periods. There are various other Stablecoins being attempted, namely Tether, Fragments, Basecoin.

Oraclize

Oraclize attempts to solve this issue by 1) delivering the data to your smart contract from an external source and 2) providing a proof that the information is from that source and unchanged. So if you trust random.org, you can use Oraclize to provide your smart contract with a random number.

Their integration is pretty powerful; you can fetch URLs, parse JSON and XPATHs, query WolframAlpha, etc.

BTCRelay

Open Zeppelin & zeppelin_os

They manage Open Zeppelin, a set of vetted smart contract best practices that you can inherit from and use in your own dApps. Check out their GitHub repo of solidity contracts for a great learning resource. Honestly you should probably just read every contract in there.

They’re taking that code reusability concept a step forward and creating zeppelin_os. Ignore the term “OS”; it’s not an operating system in a classic sense. zeppelin_os is a collection of features, tools, and services that aim to provide a solid developer experience while maximizing smart contract security.

One part of the zeppelin_os is the “zeppelin_os Kernel” which is not a kernel, but is actually a set of well-known smart contracts that act as libraries. They’re upgradeable via the proxy library model and can be independently upgraded in case of security patches. Because you’re including less code in the contract itself, deployment costs less gas and developers cut down on code repetition.

There are some other neat planned aspects of zeppelin_os, like the scheduler (async execution of contract functions, since by default contracts don’t do anything until interacted with), a marketplace protocol, and off-chain developer experience tools. You can learn more about them via the whitepaper.

* sidenote: I’m really not a fan of people adopting these already well-defined terms (like OS and Kernel) and using them to describe things that aren’t what we already understand to be an OS and a Kernel. It just adds more confusion to an already ridiculously definition-saturated space. Call your project what it is and don’t fall into the AWS naming trap that requires blog posts like this to accurately describe what you’re working on.

ENS - Ethereum Name Service

The Brave Browser & The Basic Attention Token

Brave and the Basic Attention Token was started by Brendan Eich, who originally created JavaScript and then co-founded Mozilla.

uPort & Civic

district0x

People and Players

ConsenSys

Zeppelin

Protocol Labs

Blockchain Capital Landscape

Financial Landscape

Community

Future

Finally

The purpose of this document is to create building blocks of understanding rooted in real life. None of that “key and locker” lossy metaphor nonsense. If you’ve found it to be helpful, give it a 💚 (wait no, a 👏). If you found that it did the opposite, let me know via a comment. Or on twitter dot com or whatever.

It also attempts to remove most/all of the buzzwords and high-level magic that so many people want you to believe that this technology is.

💓 self-sovereignty - XLNT.co - social-identity -Proof of Steak, Curation, UX, DX - https://mat.tc

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store