How Truffle Works Under the Hood

Ethereum dApp Development for Web Developers

Thon Ly
Thon Ly
Jul 20, 2018 · 15 min read
Photo: https://github.com/trufflesuite/truffle

Introduction

Backed by ConsenSys, Truffle is quickly becoming the premier framework for blockchain development. Like a Swiss army knife, Truffle packs together the best modules and tools to streamline smart contract creation, compilation, testing, and deployment onto Ethereum. In addition, it also supports frontend development using a Redux store that automatically syncs with the contract data. Thus, integration with React is simple and easy. For veterans who developed decentralized applications in the early days, the benefits of Truffle are obvious. For those new to dapp development, your timing is perfect!

Written in JavaScript, Truffle modularizes key features to abstract away the complexity and cognitive load. However, if we simply depend on the framework for everything, we miss out on understanding how things actually work under the hood that makes blockchain so special. Any dabbler can code by following directions. Expertise and innovation, on the other hand, require awareness of underlying assumptions and the creativity to challenge them.

Thankfully, since it’s all just JavaScript, we should be able to mimic the main functionalities of Truffle from scratch. Starting from an empty node project, we will attempt to create a smart contract, compile it, test it, and deploy it to a real ethereum network. We will also link it to an actual user interface. In so doing, the author hopes to impart a deeper understanding of the core technologies such that development using Truffle is not a dependency, but an expression of a creative choice.

Smart Contract
1. Create
2. Compile
3. Test
4. Deploy
User Interface => Smart Contract

A great primer is to have a clear mental model of the Ethereum application stack:

Our project repo: https://github.com/HeartBankAcademy/dApp

Truffle Init

Since tokens are such an integral part of the Ethereum ecosystem, let’s try to build one from scratch to better understand how Truffle works under the hood! With npm, we can begin by initializing an empty node project.

$ npm init

Then, we create empty files and folders with this architecture:

build
- Token.json
contracts
- Token.sol
migrations
- compile.js
- deploy.js
test
- Token.test.js
package.json

Like a Truffle project, the contracts folder houses all the solidity files. By convention, these files should be capitalized to indicate that solidity contracts are akin to “classes” from which contract “instances” are instantiated. In our case, it’s just Token.sol. The migrations folder houses the scripts that we use to compile and deploy our contracts. Running compile.js should compile all our solidity files into corresponding json files in the build folder. In our case, it’s just Token.json.

compile.js: contracts/*.sol => build/*.json

When we run deploy.js, the bytecode inside our json files should be deployed onto the network of our choice.

deploy.js: build/*.json => network

Finally, the test folder houses all our test files. In our case, it’s just Token.test.js.

In summary, our project directory should look like this:

This step is “equivalent” to executing:

$ truffle init

Which creates the following files and folders:

Notice the similarities! 😉

Truffle Develop

Like Remix, Truffle has a local blockchain we can use during development.

$ truffle develop

When we execute this command, Truffle launches a local network on port 9545 and generates a mnemonic phrase from which the first 10 external accounts and private keys are shown for our convenience.

Under the hood, Truffle uses a stripped-down version of ganache-cli (previously, TestRPC). Let’s install it globally and then launch a local network to see the similarities.

$ sudo npm install -g ganache-cli$ ganache-cli

Like Truffle, ganache-cli also generates a mnemonic phrase and displays the first 10 external accounts and private keys. By default, blocks are mined instantly.

Using the local network provided by Truffle on port 9545 is optional. If we prefer more controls over our local network, we can use ganache-cli instead by switching over to port 8545.

// truffle.jsnetworks: {
development: {
host: "127.0.0.1",
port: 8545,
network_id: "*"
}
}

For example, with ganache-cli we can create a more realistic network by specifying a blockTime of 15 seconds.

$ ganache-cli --blockTime 15

Or, if we prefer a graphical user interface, we can use the ganache desktop app by switching over to port 7545.

Now that we understand the function of ganache-cli, let’s install it as a project dependency so we can import it for use later.

$ npm install --save ganache-cli

We’re now ready to start coding our smart contract!

Truffle Compile

Tokens represent real-world assets, which we can buy with ethers. As such, the standard token contract, aka ERC20, is understandably complex. For simplicity, we can boil it down to this:

In essence, a token contract is just an “internal” public ledger. While the “external” public ledger records account balances in ethers, a token contract records account balances in “tokens”. We can achieve this by using a mapping data structure (balanceOf) with address as the key and uint256 as the value (line 4). As a map object, every “address balance in balanceOf will be initialized with 0.

ethereum blockchain => "external" public ledgertoken contract => "internal" public ledger

Through the constructor method, we can set the initialSupply of our tokens upon contract creation. For simplicity, the contract creator will initially possess all the tokens (line 6–8).

Creator: msg.senderbalanceOf[msg.sender] = initialSupply;

Thus, the distribution of tokens must begin from the contract creator. We can achieve this with a transfer method that accepts an address and a uint256 which specify to whom and how much (line 10–15).

From: msg.sendertransfer(address _to, uint256 _value)

Before decrementing the sender’s balance and incrementing the recipient’s balance by _value, we can perform require checks to ensure that the sender has sufficient tokens (line 11) and that the recipient’s new balance does not overflow (line 12).

require(balanceOf[msg.sender] >= _value);
require(balanceOf[_to] + _value >= balanceOf[_to]);
balanceOf[msg.sender] -= _value;
balanceOf[_to] += _value;

Altogether, our token contract becomes:

To compile our contract, we can use solc, which Truffle utilizes under the hood. Let’s install it as a dependency:

$ npm install --save solc

We want our compile.js script to begin by deleting everything in the build folder. That way, we can make changes to our contracts and recompile them anytime. In other words, we want to programmatically:

  1. Delete the build folder and everything inside.
  2. Recreate a new build folder.
  3. For each solidity file in the contracts folder, read its content to memory.
  4. For each solidity file in memory, compile it also to memory.
  5. For each compiled solidity file, write it to the build folder with .json as the extension.

To build paths to our files, we can use the standard path module. To read and write files, it’s easier to use the fs-extra module instead of the standard fs module. Let’s install that:

$ npm install --save fs-extra

Thus, our script should require in solc, path, and fs-extra:

const solc = require("solc");
const path = require("path");
const fs = require("fs-extra");

To build a path to the build directory, we can write:

const buildPath = path.resolve(__dirname, "../build");

To delete the build directory and create a new one in its place, we can write:

fs.removeSync(buildPath);
fs.ensureDirSync(buildPath);

To build a path to our token contract, we can write:

const tokenPath = path.resolve(__dirname, "../contracts", "Token.sol");

To read its content to memory, we can write:

const source = fs.readFileSync(tokenPath, "utf8");

To compile its content, we can call:

solc.compile(source, 1)

The second parameter tells solc the number of contract “classes” that are present in our file. In our case, it’s just the Token class. When compiled, it looks like this:

Notice that it’s a json object. We’re only interested in :Token because it contains the bytecode and interface (ABI) that we need. Thus, to compile and write to memory:

const output = solc.compile(source, 1).contracts[":Token"];

Finally, to write output to the build folder as Token.json:

fs.outputJsonSync(path.resolve(buildPath, "Token.json"), output);

Altogether, compile.js becomes:

To run our compile.js script, we can execute this command from the project root:

$ node migrations/compile.js

This step is “equivalent” to executing:

$ truffle compile

With that, we’re now ready to test our token contract!

Truffle Test

Under the hood, Truffle uses the mocha testing framework with chai for assertions. Let’s also use mocha:

$ npm install --save mocha

For simplicity, we can just use the standard assert library.

To be able to test our token contract, we need a local network and a way to connect to a running client, i.e., a provider. We can set one up with geth, but we will need to sign every transaction manually with Metamask (via port 8545). To get around this, we can use the ganache provider instead which comes with external accounts in an “unlocked” state.

ganache provider => local node => local network

Then, we need a way to interact with this provider using JavaScript. We can use web3 for this.

JavaScript => web3 => ganache provider 

Because transactions take time to process by the network, almost every web3 call is asynchronous. Though still in beta, version 1.0 of web3 supports Promises and therefore async and await which make code so much easier to read. Let’s install it:

$ npm install --save web3@1.0.0-beta.34

Thus, our test script begins by requiring assert, ganache-cli, and web3:

const assert = require("assert");
const ganache = require("ganache-cli");
const Web3 = require("web3");

Then, we can create an instance of Web3 with ganache.provider() as the constructor argument:

const web3 = new Web3(ganache.provider());

Now, we can use this web3 instance to instantiate our token contract. To do so, we will need the interface and bytecode from Token.json.

const { interface, bytecode } = require("../build/Token");

To connect web3 with the interface, we can write:

await new web3.eth.Contract(JSON.parse(interface))

To deploy the bytecode, we can utilize the deploy method:

.deploy({
data: bytecode,
arguments: [INITIAL_SUPPLY]
})

We can also pass in our constructor arguments to arguments as an array. To actually send the transaction, we can utilize the send method:

.send({ from: accounts[0], gas: 1000000 })

We use .send() whenever we want to update our contract’s state. Such actions alway return the transaction hash except when deploying. Available parameters are from, gas, gasPrice, and value. The originating address (from) is required but the rest is optional (default values come from ganache). gas and gasPrice specify the gas limit and unit cost of gas, while value specifies how much wei to transfer. If we only want to read data on our contract, we use .call() instead. Reading data is always free.

.send() => update contract's state // not free
.call() => read contract's state // free

As we’ve seen, ganache comes with 10 external accounts for us to use. We can access them through:

accounts = await web3.eth.getAccounts();

Thus, to deploy a fresh instance of our contract before each unit test, we can use the following test harness:

Here, we’ve arbitrarily chosen to deploy the contract from the first external account: accounts[0]

We’re now ready to write some unit tests! To begin, we should test that our token contract has indeed been deployed onto the local network. Whenever deployment is successful, web3 returns an instance of the contract that looks like this:

Notice that we can access all our public variables and functions through the methods property. Notice also that under options, the address property contains the address to our contract instance on the network. We can use this fact to make an assertion that address must exist for the test to pass.

it("deploys a contract", () => {
assert.ok(token.options.address);
});

For our second test, let’s verify that the contract creator does indeed have possession of all the tokens upon contract instantiation. Through the methods property, we can call balanceOf and pass it accounts[0].

it("has initial supply", async () => {
const supply = await token.methods.balanceOf(accounts[0]).call();
assert.equal(supply, INITIAL_SUPPLY);
});

For our third test, let’s verify that our transfer method actually behaves as we intended.

From: accounts[0]
To: accounts[1]
Amount: TOKENS
Result: balance of accounts[0] and accounts[1] should update

To make this transfer, we can call our transfer method through the methods property like so:

await token.methods.transfer(accounts[1], TOKENS)
.send({ from: accounts[0] });

To check that the transfer succeeded, we make two more calls to get the new balance of accounts[0] and accounts[1].

const fromBalance = await token.methods.balanceOf(accounts[0]).call();const toBalance = await token.methods.balanceOf(accounts[1]).call();

Then, we can verify that their balances have been appropriately updated:

it("can transfer tokens", async () => {
const TOKENS = 10;
const txHash = await token.methods
.transfer(accounts[1], TOKENS)
.send({ from: accounts[0] });
const fromBalance = await token.methods.balanceOf(accounts[0]).call();
const toBalance = await token.methods.balanceOf(accounts[1]).call();
assert.equal(fromBalance, INITIAL_SUPPLY - TOKENS);
assert.equal(toBalance, TOKENS);
});

For our last test, let’s verify that the sender needs to have a sufficient balance.

From: accounts[1]
To: accounts[0]
Amount: 10
Result: should fail

To do this, we can use a try/catch statement to catch the error and assert that it must exist:

it("requires sufficient balance", async () => {
try {
await token.methods.transfer(accounts[0], 10).send({ from: accounts[1] });
assert.fail();
} catch (error) {
assert.ok(error);
}
});

Putting all our unit tests together, we have:

To run our tests with npm, we need to add mocha to our package.json file:

"scripts": {
"test": "mocha"
}

Now, we can easily execute:

$ npm run test

This is “equivalent” to executing:

$ truffle test

With all tests passing, we’re ready to deploy our contract to a real Ethereum network!

Truffle Migrate

During our tests, we used the ganache provider to deploy our contract to our local network. Likewise, to deploy our contract to the public Ethereum network, we need a provider that connects to a real Ethereum node. We could set one up locally with geth, but for simplicity, we will use Infura instead. Also backed by ConsenSys, Infura provides endpoints that we can utilize to easily create a provider.

smart contract => ganache provider => local networksmart contract => Infura provider => public network

To create a custom provider, we can use truffle-hdwallet-provider, which Truffle utilizes under the hood:

$ npm install --save truffle-hdwallet-provider

Infura provides endpoints to the main network as well as all the test networks. Let’s use the Rinkeby testnet so we don’t have to spend real money. Let’s also create a new vault on Metamask for testing. With the Rinkeby endpoint from Infura and the seed phrase from Metamask, we can create a provider like this:

const Provider = require("truffle-hdwallet-provider");const provider = new Provider("pepper stable ripple enrich provide    business ankle tank net lumber acquire earn", "https://rinkeby.infura.io/DPHGLx2mBJeWsuDv1jFV"
);
// seed phrase and endpoint only needed until we deploy and attain the address to our contract on the blockchain

Like before, we can pass our provider to web3 to get a fresh instance:

const Web3 = require("web3");const web3 = new Web3(provider);

Reusing the deploy logic from our test harness with this web3 instance, we get:

Under the hood, Metamask also uses Infura! For reference, we output the contract address to the console. To run this script, we execute:

$ node migrations/deploy.js

Our token contract is now live on Rinkeby!

https://rinkeby.etherscan.io/address/0x6431505264aEF4CfEA510633C8Ce5970404C821f

This is “equivalent” to executing:

$ truffle migrate

Finally, let’s create a frontend that connects to our live contract!

Truffle Unbox Drizzle

To setup the boilerplate for our frontend, we can use Truffle to “unbox” Drizzle.

$ truffle unbox drizzle

Drizzle is awesome because it comes with a Redux store that syncs with our contract state.

react dapp => redux store => contract state

To better understand how this process works, we will use create-react-app instead, which Drizzle utilizes under the hood. Let’s install it globally:

$ sudo npm install -g create-react-app

And then run the create-react-app command to create a boilerplate project called “reactapp”:

create-react-app reactapp

To launch this app, we execute this command from the project root:

$ npm run start

Let’s update the main App component in src/App.js to show the balance and a form to transfer tokens:

We want our users to be able to use Metamask to sign and submit transactions to our live contract. We can achieve this by taking advantage of the web3 object that Metamask injects to the global window:

This web3 object comes with a built-in provider that we can use:

window.web3.currentProvider

However, notice that this web3 version is 0.20.3. To use version 1.0 instead, we need to install it as a dependency:

$ yarn install --save web3@1.0.0-beta.34

Then, we can take advantage of Metamask’s provider like this:

import Web3 from "web3";const web3 = new Web3(window.web3.currentProvider);

By using our version of web3, we also ensure that it will not suddenly change and break our code. Understandably, Metamask plans to stop injecting the web3 object after realizing that most dapps only need its provider. In the near future, ethereumProvider will be injected instead.

dapp => web3 => ethereumProvider => live contract

Let’s export our web3 instance for other files to use:

To connect to our live contract, we need the ABI from our compilation process and the address of our contract on Rinkeby:

const token = new web3.eth.Contract(ABI, ADDRESS);

Copying over the ABI from Token.json and the contract address from running deploy.js, we can export our live contract instance like so:

Now, we can import our token instance and web3 instance for our App component to use:

In the componentDidMount lifecycle method, we can make a web3 call to get all the user’s Metamask accounts (line 12):

const accounts = await web3.eth.getAccounts();

Then, we can call our token contract to get the balance of the first account (line 13):

const balance = await token.methods.balanceOf(accounts[0]).call();

In Metamask, the currently selected account is always accounts[0].

To help process the form submittal, we can create a helper method that calls our contract’s transfer function (line 18–32). Note that we don’t have to specify the gas and gasPrice because Metamask will automatically calculate the best values for the user.

form submit => react transfer method => contract transfer function

Because transactions take time to process on a real network, we should provide a status message to inform the user:

this.setState({ status: "Transfer in progress..." });

Let’s test the UI! When we choose the Metamask account that created the token contract (Account 1), we see the right balance of 1000, our initial supply.

Selecting a different account shows the appropriate balance of 0. Let’s now make a transfer of 50 tokens from “Account 1" to “Account 2”. Go to Metamask and copy the second account’s address:

Now past it into the form field:

When we click “Transfer”, Metamask will open a modal to ask for confirmation:

Even if no ethers are being sent, Metamask will always ask for confirmation because transactions are not free. After the network accepts our transaction, the user’s balance should update:

If we switch to “Account 2” on Metamask, the balance should update too:

Using the txHash that was returned, we can view our transaction in realtime on any blockchain explorer:

https://rinkeby.etherscan.io/tx/0x2c081e0a67545b0e92a7e19eb156a446a5431aa3c2440a029556994dde49944d

Manual testing of the UI can be quite tedious. To help us automate testing of React components, create-react-app integrates with the Jest testing framework. In fact, Drizzle goes one step further, integrating with Redux to give us a store that syncs with our contract data.

React component => Drizzle store => contract state

Moreover, Drizzle automatically generates forms for all our contract functions, which we can easily map to the appropriate reducers.

form submit => Drizzle reducer => contract function

Since the reducers are mostly asynchronous web3 calls, Drizzle also integrates with redux-saga to make asynchronous testing a breeze!

Conclusion

What makes Ethereum special is its emphasis on the smart contract layer, which Truffle also emphasizes, providing a convenient framework and a higher-order abstract contract that decorates any Solidity contracts with the functionalities to compile, test, upgrade, and deploy, seamlessly. This article is a deep dive into those features with the hope of imparting a clearer understanding and a better intuition.

For a real live example of how Truffle is used in one of our open source projects, check out:

In this article, we open up the entire source code of our dApp to explain our entire thought process, from architecture to a fully functional UI across our entire development stack!

We like to think of HeartBank® as the nonprofit twin of ConsenSys 😇

Reference: https://truffleframework.com/docs/truffle/quickstart

HeartBank Academy

Ethereum dApp Development

Thon Ly

Written by

Thon Ly

Founder @HeartBank

HeartBank Academy

Ethereum dApp Development

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade