Deploy ‘fabcar’ on Ethereum
Adventures in Blockchain
I’m spending time in November improving my knowledge of Blockchain technologies (and growing a moustache for Movember). I’m most familiar with Hyperledger Fabric but, this week, I’ve been learning Ethereum.
I tried porting (only somewhat successfully — feedback sought) Fabric’s “fabcar” sample to Ethereum. Ethereum is clearly a success but it was challenging for me to get-going. Hopefully this post will help you avoid some of my mistakes.
After this (!), I’ll back-post last week’s (yes, it wasn’t November) experience with Fabric which was more straightforward.
Ethereum
I’m running everything on an Ubuntu 16.04 VM on Google Cloud Platform (GCP). If you don’t have a GCP account, you can get started for free here. Ethereum supports Linux, Windows and Mac so, go with your favorite but, if you want reproducibility (Carl — that a word?), please match my instructions.
Assuming (!) that you are beginning on a Linux machine too, that you have Google’s Cloud SDK (the CLI for GCP) installed and you know your GCP billing account ID. The following will create a project, enable Compute Engine, create an instance called ‘ethereum-01’ and ssh into it:
BILLING=[[YOUR-BILLING-ID]]
PROJECT=[[YOUR-PROJECT-ID]]
ZONE=[[YOUR-PREFERRED-ZONE]] // us-west1-c
INSTANCE=ethereum-01gcloud alpha projects create $PROJECTgcloud alpha billing projects link $PROJECT --account-id=$BILLINGgcloud service-management enable compute.googleapis.com --project=$PROJECTgcloud compute instances create ${INSTANCE} \
--project=${PROJECT} \
--zone=${ZONE} \
--machine-type=custom-2-8192 \
--image=ubuntu-1604-xenial-v20171011 \
--image-project=ubuntu-os-cloud \
--boot-disk-size=50 \
--boot-disk-type=pd-standard \
--boot-disk-device-name=${INSTANCE}gcloud compute ssh ${INSTANCE} \
--zone=${ZONE} \
--project=${PROJECT}
Aside: If you don’t know your GCP billing account ID, you can enumerate your billing accounts with the following. The BILLING environment variable above must be one of the billing IDs from this list.
gcloud alpha billing accounts listI recommend that you duplicate the ssh command across 3 terminals in order to follow the remainder of the terminal. Everything hereon will be run in one of these terminals on ‘ethereum-01’
Ethereum CLIs
There are multiple implementations of the Ethereum CLI. I used “geth”. It is implemented in Golang and it works very well:
sudo apt-get install software-properties-common
sudo add-apt-repository -y ppa:ethereum/ethereum
sudo apt-get update
sudo apt-get install ethereumAnd then:
geth versionGeth
Version: 1.7.2-stable
Git Commit: 1db4ecdc0b9e828ff65777fb466fc7c1d04e0de9
Architecture: amd64
Protocol Versions: [63 62]
Network Id: 1
Go Version: go1.9
Operating System: linux
GOPATH=
GOROOT=/usr/lib/go-1.9
Ethereum “test network”
Ethereum runs on public networks. To avoid having to participate on one of these, to avoid complications, and to avoid having to mine Ethereum’s currency (Ether), it’s best to run a test network:
geth --dev --ipcpath ${HOME}/Library/Ethereum/geth.ipcAll being well, this should result in an ongoing (don’t kill it) console session:
INFO [11-03|18:46:19] Starting peer-to-peer node instance=Geth/v1.7.2-stable-1db4ecdc/linux-amd64/go1.9
INFO [11-03|18:46:19] Allocated cache and file handles database=/tmp/ethereum_dev_mode/geth/chaindata cache=128 handles=1024
INFO [11-03|18:46:19] Initialised chain configuration config="{ChainID: 1337 Homestead: 0 DAO: <nil> DAOSupport: false EIP150: 0 EIP155: 0 EIP158: 0 Byzantium: 0 Engine: ethash}"
WARN [11-03|18:46:19] Ethash used in test mode
INFO [11-03|18:46:19] Initialising Ethereum protocol versions="[63 62]" network=1
INFO [11-03|18:46:19] Loaded most recent local header number=5256 hash=39d6fb…ea8e29 td=2545243468
INFO [11-03|18:46:19] Loaded most recent local full block number=5256 hash=39d6fb…ea8e29 td=2545243468
INFO [11-03|18:46:19] Loaded most recent local fast block number=5256 hash=39d6fb…ea8e29 td=2545243468
INFO [11-03|18:46:19] Loaded local transaction journal transactions=11 dropped=11
INFO [11-03|18:46:19] Regenerated local transaction journal transactions=0 accounts=0
WARN [11-03|18:46:19] Blockchain not empty, fast sync disabled
INFO [11-03|18:46:19] Starting P2P networking
INFO [11-03|18:46:19] RLPx listener up self="enode://f10855ec7cc692a8276276fa08ae6faef898d69f9063357b49814b2b28a436bce432dc98a1609f690c5852c64d1b34dcae3a3fe984036c78ba2804a142ddc24e@[::]:40559?discport=0"
INFO [11-03|18:46:19] started whisper v.5.0
INFO [11-03|18:46:19] IPC endpoint opened: /home/dazwilkin/Library/Ethereum/geth.ipcI’ll refer to this terminal session as the “console”.
Please create another ssh session on ‘ethereum-01’. From this session, we’ll run the CLI session against which we’ll issue commands against our Ethereum private network. From this, second ssh session on ‘ethereum-01’, type:
geth attach ipc://${HOME}/Library/Ethereum/geth.ipcNB: this makes calls ‘attach’ using geth and references the same IPC that we’re using on the console.
All being well, you should see:
Welcome to the Geth JavaScript console!instance: Geth/v1.7.2-stable-1db4ecdc/linux-amd64/go1.9
coinbase: 0x3c708ddabda3ad31afd8ec7482ba3afc22b8314c
at block: 5256 (Fri, 03 Nov 2017 18:45:59 UTC)
datadir: /tmp/ethereum_dev_mode
modules: admin:1.0 debug:1.0 eth:1.0 miner:1.0 net:1.0 personal:1.0 rpc:1.0 shh:1.0 txpool:1.0 web3:1.0>
We are now in the Ethereum JavaScript console.
I’ve yet to find the ‘help’ equivalent. Feel free to explore this environment. I recommend you start by creating an account:
> personal.newAccount("[[YOUR PASSWORD]]")Because we’re in a JavaScript console, if you’re familiar with JavaScript, Node.js, you should quickly find your way around. For example, the above uses the ‘personal’ object, so let’s display the object:
> personal
{
listAccounts: ["0x3c...","0x00..."],
listWallets: [{
accounts: [{...}],
status: "Locked",
url: "keystore:///tmp/..."
}, {
accounts: [{...}],
status: "Locked",
url: "keystore:///tmp/..."
}],
deriveAccount: function(),
ecRecover: function(),
getListAccounts: function(callback),
getListWallets: function(callback),
importRawKey: function(),
lockAccount: function(),
newAccount: function github.com/ethereum/go-ethereum/console.(*bridge).NewAccount-fm(),
openWallet: function github.com/ethereum/go-ethereum/console.(*bridge).OpenWallet-fm(),
sendTransaction: function(),
sign: function github.com/ethereum/go-ethereum/console.(*bridge).Sign-fm(),
unlockAccount: function github.com/ethereum/go-ethereum/console.(*bridge).UnlockAccount-fm()
}In my case, I have created 2 accounts but you will likely only see one. You can see that these accounts are added to an array called personal.listAccounts and there are an equal number of personal.listWallets. You can access these through functions: personal.getListAccounts() and personal.getListWallets() too.
These hex values (prefixed “0x”) are Ethereum transaction IDs. You should see the same transaction ID return when you unlock your account.
Another function is called personal.unlockAccount. We’ll use it frequently:
> personal.unlockAccount(eth.accounts[0])Unlock account 0x3c...
Passphrase: [[YOUR PASSWORD]]
true
Before deploying Contracts (see below) and any other time you’re told you’re not authenticated, please run this command and provide your password.
One very important command that you should issue — I learned this the hard way — is to set your default account:
> eth.defaultAccount=eth.coinbase;This should return the Ethereum transaction ID hash for your personal account (from above). We are now good to go!
Ethereum Contracts
My apologies if I mangle Ethereum’s vernacular here. I’ll use the terms as I think they apply and I’ll correct as people educate me…
Ethereum’s Contracts are written in a language called Solidity. It’s derived from JavaScript but it has some very nice features including Structs, Mappings and types (!). I had most of my challenges learning Ethereum with Solidity. It feels ‘pernickety’.
As I mentioned previously, I thought it would be useful to port Fabric’s “fabcar” example to Ethereum so that I could compare both platforms more like-for-like. I’ve not been able to reproduce the functionality exactly and I’ll explain why as we continue but I got to a workable subset.
Cutting to the chase, here’s what I have. Mostly so as not to confuse myself, it’s called “Simple” rather than “Fabcar” and should be saved in a file called simple.sol:
pragma solidity ^0.4.0;contract Simple {struct Car {
string make;
string owner;
}mapping (string => Car) cars;function Simple()
public
{
createCar("CAR0","Ford","Tomoko");
createCar("CAR1","Toyota","Brad");
createCar("CAR2","Hyundai","Jin Soo");
}
function createCar(string key, string make, string owner)
public
payable
{
cars[key] = Car(make,owner);
}
function changeCarOwner(string key, string newOwner)
public
payable
{
cars[key].owner = newOwner;
}
function queryCar(string key)
public
view
returns (string, string)
{
return (cars[key].make, cars[key].owner);
}
}
I won’t reproduce Solidity’s documentation here but here are the most important observations:
- Everything is contained within a single Contract called “Simple”.
- In fabcar, Cars have colour, make, model and owner. For some (as yet unexplained reason), I can only get this contract to work if Cars have only 2 properties. Because we want to implement changeCarOwner, I’m going with “make” and “owner” properties.
- Cars is defined as a (self-explanatory) Struct comprising the aforementioned two (in this case: string) properties.
- Mapping is a data-type that always accepts any value for the key type and returns the value type. In this case, per fabcar, I’m using a string key (CARx) and, our value type is an instance of the Car Struct.
- Whereas fabcar uses initLedger, with Solidity we can create a constructor for the Contract by specifying a function named the same as the Contract (“Simple”). This, as you’d expect, calls “createCar”.
- createCar takes a key (CARx) and our only two properties: ‘make’ and ‘owner’ and creates a mapping between the key and a Car Struct created with the ‘make’ and ‘owner’ values.
- changeCarOwner does not check whether the key (CARx) is valid. This is partly laziness and partly because of the way Mapping is implemented. Every key exists and is valid. The check could be (!) to check whether Car owner is “” but…
- queryCar returns the Car values (make, owner) for any key that is given to it. If the key is invalid, e.g. “X”, then it will still return a result albeit empty (“”) strings for both properties.
- It’s not obvious how I’d implement queryAllCars. Mappings seems a more idiomatic way to persist Cars in Ethereum but it would be possible to use a dynamic array too. Perhaps that’s the better solution?
Let’s deploy!
Deploying using Remix
Ethereum provides a browser-based Solidity compiler called “Remix”. I found this to be very useful when I was having Solidity problems because it removed any uncertainty that I was compiling Solidity incorrectly. So, I’ll show you Remix first and then how you can do this using the “solc” compiler.
Visit:
https://ethereum.github.io/browser-solidityYou will be presented with a default Contract (Ballot). If you wish to proceed with “Simple”, delete the “ballot.sol” file and, using the “+” icon in the top left-hand corner, create a new file called “simple.sol”, paste the contents of the Simple “sol” file that I included above.
You can click “Start to compile” although it should compile automatically. Look for the green box on the right hand side “browser/simple.sol:Simple”. The file is called “simple.sol”. The contract is called “Simple”.
Click the “Details” button:
The 5th box is called “WEB3DEPLOY” and it has a “copy-to-clipboard” icon, click it. Now, return to the ssh session on ethereum-01 that contains the Ethereum JavaScript Console.
Unlock your personal account:
> personal.unlockAccount(eth.accounts[0])And then paste (CTRL-V) the contents from the clipboard (web3deploy) into the session. The session should report:
null [object Object]
undefinedYou may (!) need to complete a follow-up step. If you do not see something like this:
Contract mined! address: 0x962b86d1b84d32520e99efce70ccd483befbe362 transactionHash: 0x0bbcbeec330a88b75630151d40712f3b2d6b8e8ac103ce22fbf4095a8d497a9bThen, please type:
> miner.start()and, after a few seconds, you should see the “Contract mind!” response.
Remix creates a rather lengthy name for our Contract (browser_simple_sol_simple). In the next step, we’ll use something shorter. But, for now, let’s use this to be clear what we’re using.
As before, “browser_simple_sol_simple” is a JavaScript object, so:
> browser_simple_sol_simple{
abi: [{
constant: false,
inputs: [{...}, {...}, {...}],
name: "createCar",
outputs: [],
payable: true,
stateMutability: "payable",
type: "function"
}, {
constant: true,
inputs: [{...}],
name: "queryCar",
outputs: [{...}, {...}],
payable: false,
stateMutability: "view",
type: "function"
}, {
constant: false,
inputs: [{...}, {...}],
name: "changeCarOwner",
outputs: [],
payable: true,
stateMutability: "payable",
type: "function"
}, {
inputs: [],
payable: false,
stateMutability: "nonpayable",
type: "constructor"
}],
address: "0x962b86d1b84d32520e99efce70ccd483befbe362",
transactionHash: "0x0bbcbeec330a88b75630151d40712f3b2d6b8e8ac103ce22fbf4095a8d497a9b",
allEvents: function(),
changeCarOwner: function(),
createCar: function(),
queryCar: function()
}
NB: that the “abi” property includes “createCar”, “queryCar”,”changeCarOwner” of type “function” and one type “constructor”. And that these functions are represented as functions on the object itself.
OK… Let’s hit it:
> browser_simple_sol_simple.queryCar("CAR0")
["Ford", "Tomoko"]Great.
> browser_simple_sol_simple.changeCarOwner("CAR0","Henry")
Error: authentication needed: password or unlock
at web3.js:3143:20
at web3.js:6347:15
at web3.js:5081:36
at web3.js:4137:16
at apply (<native code>)
at web3.js:4223:16
at <anonymous>:1:1Hmmm…. As mentioned before, you’ll likely encounter this frequently. The solution is to:
personal.unlockAccount(eth.accounts[0])Then retry:
> browser_simple_sol_simple.changeCarOwner("CAR0","Henry")
"0x3e96e35994006302396b933466f18a190cb6788fd1b9a1975c1bfe271fe2d3ce"> browser_simple_sol_simple.queryCar("CAR0")
["Ford", "Tomoko"]
Hmmm… That’s not correct. I changed the owner of “CAR0” to “Henry” but, when then querying “CAR0”, it still reports “Tomoko” as the owner. This “eventual consistency” is an innate behavior with Blockchains. Try it again:
> browser_simple_sol_simple.queryCar("CAR0")
["Ford", "Henry"]Now, the Car has been updated. We don’t (ever) know whether our query results are outdated (Is “Henry” still the owner of “CAR0”? We don’t know). But this is just the way it is.
In our test network, you can force this behavior to be more transparent because you control the network entirely.
> miner.stop()Now that mining is stopped, transactions can’t be committed to the Blockchain. Let’s try the command again:
> browser_simple_sol_simple.changeCarOwner("CAR0","Daz")
"0xd54d35175e8d586924fe00616b89a6a34b78a4a3d8197325e342907d2040ece2"> browser_simple_sol_simple.queryCar("CAR0")
["Ford", "Henry"]> browser_simple_sol_simple.queryCar("CAR0")
["Ford", "Henry"]> browser_simple_sol_simple.queryCar("CAR0")
["Ford", "Henry"]> browser_simple_sol_simple.queryCar("CAR0")
["Ford", "Henry"]
We can continue to check “CAR0” for eternity and its owner property will stay as “Henry”.
If you switch to the first terminal that shows the Blockchain logging, you should see something similar to:
INFO [11-03|20:04:25] Submitted transaction fullhash=0xd54d35175e8d586924fe00616b89a6a34b78a4a3d8197325e342907d2040ece2 recipient=0x962b86D1B84D32520E99eFce70CCd483BEfBE362The hash of the submitted transaction (in my case “0xd54d35…”) corresponds to the hash presented to us from the changeCarOwner command. But, it has not yet been mined to a block because we paused mining.
Flip to the CLI session and resume then promptly stop mining:
> miner.start()
> miner.stop()And back to the logging session:
INFO [11-03|20:05:30] Starting mining operation
INFO [11-03|20:05:30] Commit new mining work number=5601 txs=1 uncles=0 elapsed=594.603µs
INFO [11-03|20:05:31] Successfully sealed new block number=5601 hash=56e27d…321305
INFO [11-03|20:05:31] 🔗 block reached canonical chain number=5596 hash=f9d501…a7b5d7
INFO [11-03|20:05:31] 🔨 mined potential block number=5601 hash=56e27d…321305
INFO [11-03|20:05:31] Commit new mining work number=5602 txs=0 uncles=0 elapsed=239.941µsAnd now, we should be able to get the update owner of CAR0:
> browser_simple_sol_simple.queryCar("CAR0")
["Ford", "Daz"]Lastly, let’s:
> browser_simple_sol_simple.createCar("CAR3","Porsche","Dreams")
"0x4e4a96bea684f6bfa3a4675765d31b0ae50acee92ae923a9b6de379d14f27241"> browser_simple_sol_simple.queryCar("CAR3")
["", ""]> browser_simple_sol_simple.queryCar("CAR3")
["", ""]
You should know why this doesn’t show the new car but you may not know why it returns a result. This is because Solidity Mapping objects have values for every (possibly) key regardless of whether the key has been explicitly added.
> miner.start()
null> browser_simple_sol_simple.queryCar("CAR3")
["", ""]> browser_simple_sol_simple.queryCar("CAR3")
["Porsche", "Dreams"]> miner.stop()
true
Deploying the harder way
There are 6 steps to deploying contracts to an Ethereum network:
- Install the Solidity compiler (solc)
- Compiling Solidity to JavaScript
- Loading the JavaScript in the network
- Munging (!) the JavaScript
- Determining the gas value to run the Contract
- Running the Contract
The “gas” value is an internal unit of work. I believe (!) there’s a fixed valuation between “gas” and Ethereum “Ether” (the currency).
Install the Solidity compiler (solc)
http://solidity.readthedocs.io/en/develop/installing-solidity.htmlAnd on Linux the instructions are:
sudo add-apt-repository ppa:ethereum/ethereum
sudo apt-get update
sudo apt-get install solcCompiling Solidity to JavaScript
From the 3rd (and as yet unused) ssh session on ethereum-01, create a file called “simple.sol” and paste the contents from above in to it.
Run the following command:
echo var contractContent = > simple.js
solc --combined-json abi,bin,interface simple.sol >> simple.jsThe creates a JavaScript program (simple.js) that encapsulates the Solidity Contract. The relevant pieces are added to a JavaScript object called “contractContent”.
Load the JavaScript into the network
Switch to the CLI session and:
> loadScript("simple.js")
trueOptional: If you wish:
> contractContentMunging the JavaScript & Determining the gasValue
Then:
> var contractTemplate=web3.eth.contract(JSON.parse(contractContent.contracts["simple.sol:Simple"].abi))
undefined> var gasValue = eth.estimateGas({data:"0x"+contractContent.contracts["simple.sol:Simple"].bin})
undefined> gasValue
763187
The first command of these 3 is the only one that is necessary. The gasValue is interesting. When we used Remix, it provided a gasValue (in my case) of ‘4700000’. The computed value here is ‘763187’ which is much lower. I’d recommend using the value used by Remix (it worked). So:
> gasValue=4700000Run the Contract
Then:
var simple = contractTemplate.new({
from: eth.accounts[0],
data: "0x" + contractContent.contracts["simple.sol:Simple"].bin,
gas:gasValue
},
function (e, contract) {
console.log(e, contract);
if (typeof contract.address !== 'undefined') {
console.log('Contract mined! address: ' + contract.address + 'transactionHash: ' + contract.transactionHash);
}
}
);This mirrors the web3deploy functionality and uses a boilerplate (anonymous) function that produces the “Contract mined!” message.
All being well, as before:
Contract mined! address: 0x023ccb7d6475eb59ccafb4eed49d72caf77be111transactionHash: 0x48c1681b8acd3ad0cafdf7c5bb8e4070034661fc02f45672510744815b869248If you don’t get this success message, you may need to start mining. Once you receive it, as before but now simply “simple”:
> simple.queryCar("CAR0")
["Ford", "Tomoko"]This is a distinct application and so you should not expect to see the Car Structs that you created previously but, we may recreate them:
> simple.createCar("CAR3","Porsche","Dreams")
"0x85ea30f48f3b4a7b9ec16d437d3550a950d8531b34bbb095de2387013da444dd"> simple.queryCar("CAR3")
["", ""]> miner.start()
null> simple.queryCar("CAR3")
["", ""]> simple.queryCar("CAR3")
["Porsche", "Dreams"]> miner.stop()
true
Conclusions
Ethereum is a compelling Blockchain offering. Solidity appears to be a good language in which to write Contracts although I’ve been stymied by several issues and struggled with poor documentation.
When having problems deploying Contracts, I tried using pyethapp instead of geth and tried to use the konradkonrad/docker-pyeth-cluster. I was unsuccessful with the Docker-based test network and gave up.
Tear-down
To exit from the CLI session, type “exit”
To exit from the logging session, type “CTRL-C”:
INFO [11-03|20:36:37] Got interrupt, shutting down...
INFO [11-03|20:36:37] IPC endpoint closed: /home/dazwilkin/Library/Ethereum/geth.ipc
INFO [11-03|20:36:37] Blockchain manager stopped
INFO [11-03|20:36:37] Stopping Ethereum protocol
INFO [11-03|20:36:37] Ethereum protocol stopped
INFO [11-03|20:36:37] Transaction pool stopped
INFO [11-03|20:36:37] Database closed database=/tmp/ethereum_dev_mode/geth/chaindata
INFO [11-03|20:36:37] whisper stoppedYou may “exit” from all the ssh sessions.
You should be returned to your host session. The one from which you ran the gcloud commands. You may either stop the ethereum-01 VM:
gcloud compute instances stop ${INSTANCE} \
--zone=${ZONE} \
--project=${PROJECT}Or (permanently) delete it:
gcloud compute instances delete $(INSTANCE} \
--zone=${ZONE} \
--project=${PROJECT}
--quietOr simply delete the entire GCP project:
gcloud alpha projects delete ${PROJECT} --quiet