How to Manage Your Full Nodes — Part 1: Containerizing Bitcoin and Ethereum with Docker
This article is the part 1 of 3 series on How to manage your full nodes.
Today, we will be talking about — Part 1: Containerization Bitcoin and Ethereum using Docker.
Why run a full a node?
Learning Objectives
- Write a simple Dockerfile for Bitcoin and Ethereum
- Deploy Bitcoin and Ethereum with Docker
Benefits of Containerization
Isolation
Every container has it’s own dedicated CPU, memory and network resources, meaning we can run multiple blockchain clients on the same machine without worrying if the one client’s excess resource consumption would cause the other clients to crash.
Security
Isolation means that each container is unable to view other containers and the host’s running processes and resources, which reduces the surface area for attackers. Damage done to a blockchain client running inside the container is mostly limited to that container.
Continuous Deployment
Write once , deploy everywhere. Whether we are upgrading the client version or adding a new blockchain client to the machine altogether, we will produce a new container image which can be pushed to external Docker repositories like Amazon (ECR) , Dockerhub for easy access. It also means that we are agnostic to the host operating system / cloud provider so long as the platform supports Docker, it can run our application. This combined with CI / CD tools like CircleCI will make deployment to production environments much faster.
Standardizing Environments
In the past, one of the largest peeves among developers is for an application to run well on their local machine only to fail in testing or production environments. Containers ship with their dependencies explicitly stated and packaged with the application, with environment specific configuration either laid out in a configuration file or coded into the environment variables. Each container image is given a hash, which can be used to manage application releases allowing for easy rollback / implementation of multiple versions of a blockchain client.
What is Docker?
Docker is a tool that allows you to create, deploy and manage applications in containers. Containers contain everything required to run the application ranging from packaged libraries to environment variables and much more. Each container is isolated from the host machine and is deployed with it’s own CPU, memory, and network.
For a great introduction to Docker concepts check out Preethi Kasireddy’s article at https://medium.freecodecamp.org/a-beginner-friendly-introduction-to-containers-vms-and-docker-79a9e3e119b.
Prerequisites
Download blockchain clients
Official Bitcoin Core client:
Choose the Linux (tgz) option.
Instructions on how to verify the clients with Bitcoin Core’s release signature can also be found on that page.
Official Ethereum Client:
Choose the Geth 1.8.12 for Linux.
Setting Up Docker
You may find relevant instructions for installing Docker and Kubernetes for your operating system below:
Writing the Configs and Dockerfiles
Bitcoin
client.conf
##
## bitcoin.conf configuration file. Lines beginning with # are comments.
### Uncomment below to run on the test network instead of the real bitcoin network.# testnet=0# server=1 tells Bitcoin-Qt and bitcoind to accept JSON-RPC commands
server=1# Data Directory
datadir=/app/data
# Index Transactions
txindex=1
# RPC
rpcbind=0.0.0.0:8332
rpcallowip=::/0
In the above client.conf file, we specify a custom data directory /app/data, start a JSON-RPC server and index the transactions, so that old spent transactions can be found by the getRawTransaction RPC call.
Dockerfile
# 1. Base OS Image
FROM ubuntu:16.04# 2. Copy blockchain client
ADD bitcoin-0.16.2 /app# 3. Setup directory
WORKDIR /app/bin# 4. Make permissions
RUN chmod +x /app/bin/bitcoind# 5. Add client.conf file
COPY client.conf /app/client.conf# 6. Entrypoint
ENTRYPOINT ["/app/bin/bitcoind", "-conf=/app/client.conf"]
We use Ubuntu Server 16.04 LTS as our base image, copy the bitcoin client folder to /app/ and set the permissions on the bitcoin client executable.
Finally, we copy the configuration file into the /app/ directory and set the ENTRYPOINT to run the Bitcoin client.
Your File Directory should look like this:
> Your Folder
> bitcoin-0.16.2 (Bitcoin Client Folder from Bitcoin Website)
> client.conf
> Dockerfile*UPDATE (14 Aug 2018): The command #2 Copy blockchain client
‘ADD bitcoin-0.16.2 /app’ in the Dockerfile is dependant on the latest version that is available at https://bitcoincore.org/en/download/.*
Ethereum
Extract the downloaded .tar.gz file into a folder and rename it geth.
You can get the default geth client.toml file by running
geth dumpconfig > client.tomlYou can also customize the config by adding CLI options specified here:
https://github.com/ethereum/go-ethereum/wiki/Command-Line-Options
Below is a sample of the default config file with the database cache increased to 1536 for faster syncing.
client.toml
[Eth]
NetworkId = 1 # 1 - Main Network, 3 - Ropsten , 4 - Rinkeby
SyncMode = "fast"
LightPeers = 100
DatabaseCache = 1536 # 0.75 of 2048, Amount of memory to use.
GasPrice = 18000000000
EnablePreimageRecording = false[Eth.Ethash]
CacheDir = "ethash"
CachesInMem = 2
CachesOnDisk = 3
DatasetDir = "/app/data/.ethash"
DatasetsInMem = 1
DatasetsOnDisk = 2
PowMode = 0[Eth.TxPool]
NoLocals = false
Journal = "transactions.rlp"
Rejournal = 3600000000000
PriceLimit = 1
PriceBump = 10
AccountSlots = 16
GlobalSlots = 4096
AccountQueue = 64
GlobalQueue = 1024
Lifetime = 10800000000000[Eth.GPO]
Blocks = 20
Percentile = 60[Shh]
MaxMessageSize = 1048576
MinimumAcceptedPOW = 2e-01[Node]
DataDir = "/app/data/ethlive"
IPCPath = "geth.ipc"
HTTPHost = "0.0.0.0"
HTTPPort = 8545 # JSON RPC Port
HTTPVirtualHosts = ["localhost"]
HTTPModules = ["net", "web3", "eth", "shh"]
WSHost = "0.0.0.0"
WSPort = 8546
WSModules = ["net", "web3", "eth", "shh"][Node.P2P]
MaxPeers = 25
NoDiscovery = false
BootstrapNodes = ["enode://a979fb575495b8d6db44f750317d0f4622bf4c2aa3365d6af7c284339968eef29b69ad0dce72a4d8db5ebb4968de0e3bec910127f134779fbcb0cb6d3331163c@52.16.188.185:30303", "enode://3f1d12044546b76342d59d4a05532c14b85aa669704bfe1f864fe079415aa2c02d743e03218e57a33fb94523adb54032871a6c51b2cc5514cb7c7e35b3ed0a99@13.93.211.84:30303", "enode://78de8a0916848093c73790ead81d1928bec737d565119932b98c6b100d944b7a95e94f847f689fc723399d2e31129d182f7ef3863f2b4c820abbf3ab2722344d@191.235.84.50:30303", "enode://158f8aab45f6d19c6cbf4a089c2670541a8da11978a2f90dbf6a502a4a3bab80d288afdbeb7ec0ef6d92de563767f3b1ea9e8e334ca711e9f8e2df5a0385e8e6@13.75.154.138:30303", "enode://1118980bf48b0a3640bdba04e0fe78b1add18e1cd99bf22d53daac1fd9972ad650df52176e7c7d89d1114cfef2bc23a2959aa54998a46afcf7d91809f0855082@52.74.57.123:30303", "enode://979b7fa28feeb35a4741660a16076f1943202cb72b6af70d327f053e248bab9ba81760f39d0701ef1d8f89cc1fbd2cacba0710a12cd5314d5e0c9021aa3637f9@5.1.83.226:30303"]
BootstrapNodesV5 = ["enode://06051a5573c81934c9554ef2898eb13b33a34b94cf36b202b69fde139ca17a85051979867720d4bdae4323d4943ddf9aeeb6643633aa656e0be843659795007a@35.177.226.168:30303", "enode://0cc5f5ffb5d9098c8b8c62325f3797f56509bff942704687b6530992ac706e2cb946b90a34f1f19548cd3c7baccbcaea354531e5983c7d1bc0dee16ce4b6440b@40.118.3.223:30304", "enode://1c7a64d76c0334b0418c004af2f67c50e36a3be60b5e4790bdac0439d21603469a85fad36f2473c9a80eb043ae60936df905fa28f1ff614c3e5dc34f15dcd2dc@40.118.3.223:30306", "enode://85c85d7143ae8bb96924f2b54f1b3e70d8c4d367af305325d30a61385a432f247d2c75c45c6b4a60335060d072d7f5b35dd1d4c45f76941f62a4f83b6e75daaf@40.118.3.223:30307"]
StaticNodes = []
TrustedNodes = []
ListenAddr = ":30303"
EnableMsgEvents = false[Dashboard]
Host = "localhost"
Port = 8080
Refresh = 500000000
Dockerfile
# 1. Base OS Image
FROM ubuntu:16.04
# 2. Create the Ethereum directory
ADD geth /app
COPY client.toml /app/client.toml
# 3. Setup entrypoint
WORKDIR /app
RUN chmod +x /app/geth
# 4. Setup NTP
CMD timedatectl set-ntp no
CMD apt-get install ntp
ENTRYPOINT ["/app/geth", "--config /app/client.toml >> /app/data/stdout.log 2>> /app/data/stderr.log"]In the above Dockerfile, we use Ubuntu 16.04 LTS as the base image, add the geth directory to /app/ and copy the configuration file to the app. We have to set up NTP for the Ethereum client to work correctly.
Lastly, we set the ENTRYPOINT to run the Ethereum client to use the client.toml and pipe the STDOUT and STDERR to two files so we can look up the logs on the filesystem later.
Your file directory should look like this:
> Your Folder
> geth Folder (Downloaded from Ethereum website)
> client.toml
> DockerfileDeploying with Docker
In the above two configuration files, we specified the data directory to be /app/data. We will continue now to create two folders /data/btc-live and /data/eth-live in order to mount them as volumes into the Docker container later.
Bitcoin
docker build -t test-btc-img .docker run --name testing-btc-live -v /data/btc-live:/app/data -p 18332:8332 -p 8333:8333 -td test-btc-img
In the above command, we build a new Docker Image from the Dockerfile and tag it as test-btc-img . Next, we create a new Docker container named testing-btc-live and mount the /data/btc-live folder we have on our host as the /app/data volume in the container. Then, we expose the container ports 8332,8333 on the host as ports 18332,8333 for the JSON-RPC server and Bitcoin client respectively.
Ethereum
docker build -t test-eth-img .docker run --name testing-eth-live -v /data/eth-live:/app/data -p 30303:30303 -p 8545:8545 -td test-eth-img
We build a new Docker image from the Dockerfile and tag it as test-eth-img . Then, we create a new Docker container named testing-eth-live and mount the /data/eth-live/ folder on the host as the /app/data volume in the container. Finally, we expose the container ports 30303,8545 on the host as ports 30303,8545 for the Ethereum client and JSON-RPC server respectively.
Geth’s output logs are piped to stderr , which are now available in your /data/eth-live directory as stderr.log
Summary
We have successfully created and deployed two Docker containers for Bitcoin and Ethereum for the main net. We can also create testnet clients for Bitcoin and Ethereum by changing the configuration files specified above and exposing the port 18333 in place of 8333 for Bitcoin. We have barely scratched the surface of what Docker is able to do but I hope this short tutorial is helpful for those who are starting out running their full nodes.
Want to learn more?
Follow us on our Twitter and Medium for more updates. If you are a developer who is keen to contribute towards building a developer community alongside MW Partners, kindly reach out to hello@mwpartners.io.
P.S, a little teaser to all our readers — our next article will touch on Kubernetes, an open-source container-orchestration system for automating deployment, scaling and management of containerized application. Kindly stay tuned!
