Building a Permissioned Blockchain Pt 1:
Deploying Hyperledger Fabric Cluster with Apache Brooklyn.
Deploying your own permissioned blockchain can be a real pain and lead to poor configurations, non-working nodes, and lost time. Luckily the folks over at cloudsoft have written a solution atop the Apache Brooklyn server provisioning service. This is a quick guide on how to get quickly set up on a server
A Word About Apache Brooklyn
According to the Apache Brooklyn homepage:
“Brooklyn is a framework for modeling, monitoring, and managing applications through autonomic blueprints.”
In short, this means that Apache Brooklyn is a program that takes ‘blueprints’ or text files with configuration information and spins up servers and application based on the contents of these configuration files in the cloud. In this case, hyperledger-brooklyn will be a service deployed inside Amp.
Amp
Amp is the management software written by Cloudsoft that you will use to monitor and manage servers. It deploys on top of a Vagrant Virtual Machine instance which will provide an SSH IP for you to make direct changes to your management server. Note that you are not deploying nodes on your computer itself. Rather, you are deploying nodes on a cloud service such as IBM Softlayer, Amazon AWS EC2, or Google Compute Engine.
When paired with Virtual Machine software, vagrant, and a server platform, Brooklyn Hyperledger is a powerful tool for quickly deploying a permissioned blockchain.
Requirements
- Vagrant with a VM — Installation Information
- Cloudsoft Amp — Github
- Brooklyn Hyper Ledger Blueprints — docker — catalog
I. Getting the Amp UI started
First install Vagrant with a Virtual Machine of your choosing. We recommend VmBox with Vagrant.
Next, get the cloudsoft amp from github and run it with vagrant. Open a terminal and run the following commands:
$ git clone https://github.com/cloudsoft/amp-vagrant
$ cd amp-vagrant
$ vagrant up amp
The git command brings down the directory tree for the Amp server. You will see a set of outputs that notify you that the amp server is up. The next cd command sets the current directory to a directory in the amp-vagrant folder. Finally, the vagrant up amp asks vagrant to look for the VagrantFile and create the image for the machine. This is also the command and directory you will use to re-start and manage the vagrant instance on your local machine.
After this is successful you should be back to your command prompt with a set of messages. In that batch of text above, you can find IP address to ssh into and the server:port of your vagrant-Amp instance.
Usually, the Amp interface sits at 10.10.10.100:8081 on your local machine. By accessing this address, you also will have access to the amp user interface.
Navigate to http://10.10.10.100:8081 with your browser. You will be greeted by a log in screen. The username is ‘admin’ and the password is ‘password’
Now you should be able to navigate around the amp console. You should see this:
Amp acts as a staging and management platform for provisioning and managing servers. In order to deploy a specific application (such as hyperledger fabric) to your servers, you must install the blueprints for the applications into the Amp interface.
II. Adding the Hyperledger
Now keep in mind that even though you have Amp UI started, you don’t yet have the hyper ledger application installed on the Amp UI. The application is merely a set of scripts that the amp uses to construct, start, and manage your servers.
In order to add the application to the Amp interface click on the ‘classic ui’ button in the top right. We’ve found that the classic UI works best for adding applications. In fact, we prefer it for most deployment tasks while the newer UI is good for management. This area can sometimes hang and take a few moments to load elements so be patient.
After navigating to the classic UI, click on the ‘catalog’ tab on the upper right side of the screen. After this click the ‘+’ plus sign button next to the word ‘catalog’ on the left side. A set of buttons will appear, one says ‘YAML’ and the other, Location. Click the YAML button. From here paste this:
brooklyn.catalog:
items:
— https://raw.githubusercontent.com/cloudsoft/brooklyn-hyperledger/master/docker.bom
— https://raw.githubusercontent.com/cloudsoft/brooklyn-hyperledger/master/catalog.bom
And then click, ‘add to catalog’. After a pause you should be able to navigate to your home console. Do this. This will add the items into the catalog, adding them to your main console. Now, you should be able to see the Hyperledger Fabric on the quick launch panel like so:
If you don’t immediately see it, scroll to the right until it appears.
After you make it through this guide, we encourage you to look at the files you are using for yourself. You can modify them as well to change certain aspects of your deployment, however, it’s a good idea to get a better understanding of how the entire hyperledger-brooklyn ecosystem works. The links for these are here and here. We will be publishing a more comprehensive breakdown of the brooklyn-Apache blueprint system soon as a part of this ongoing series.
III. Setting up your Locations
Now that the application is installed on the AMP, you need a location to actually start up a server. For this guide we will be demonstrating on Amazon AWS EC2. Keep in mind that the servers you spin up for your cluster are deployed on large instances and may cost you money.
Now, first you must set up your identity information. In order to use your EC2 instance and gain root control, the AMP server must have an access ID and access key for your Amazon AWS account. You can generate this by navigating to the ‘Security Credentials’ under the settings are of your main account. You can use the IAM properties of AWS if you want but hassling with that is outside the scope of this guide. You can find out more about AWS security and IAM services here.
After getting to Security Credentials, click ‘Access Keys’ and then click the blue button labeled, “Create New Access Key”. Make sure to save both the access ID and the security secret.
Now, this next part is a bit tricky.
In a text editor, copy and paste this next part (just the text):
/////////////////////////////////////////////////////////////////////
brooklyn.catalog:
items:
- id: {NAME_ON_CONSOLE}
# NB: the version may need to be increased
version: 0.0.0.SNAPSHOT
itemType: location
item:
type: jclouds:aws-ec2
brooklyn.config:
identity: {AWS_ACCESS_ID}
credential: {AWS_SECRET_KEY}
region: {REGION}
imageId: {REGION}/{AMI_CODE}
minRam: 2000
loginUser: centos
installDevUrandom: true
allocatePTY: true
/////////////////////////////////////////////////////////////////////
We will provide a better breakdown of this later, and for now here is a legend of each needed substitution. Any entry inside brackets needs to be replaced.
First, under ‘id’, type the name you want the node to appear under in the management catalog.
Next replace {AWS_Access_ID} replace it with the access ID you generated earlier.
After that, replace {AWS_SECRET_KEY} with the key you generated earlier.
Fourth, Replace the region with the AWS-EC2 region you keep your EC2 instance. Make sure to remove the letter after the region as well. For instance, if your instance is us-west-1a, list us-west-1. Same for the other area called ‘REGION’.
Finally, under AMI_CODE, place the ami number of your region and build. Unfortunately, the ami changes based on build and type of instance so after you choose your instance you may need to actually spin up an instance to grab this code. We recommend the “CentOS 7 (x86_64) with Updates HVM” which can be found here.
This is the blueprint for your location. It will notify the AMP which area and service you wish to deploy a server. Here’s an example of a completed one for a Tokyo node:
/////////////////////////////////////////////////////////////////////
brooklyn.catalog:
items:
- id: aws-tokyo-centos7
# NB: the version may need to be increased
version: 0.0.0.SNAPSHOT
itemType: location
item:
type: jclouds:aws-ec2
brooklyn.config:
identity: AKIAJ4ADSR7FSFFSSA
credential: jsdfjdfkjFJFr9f/fjdfFDjfeiefsdfsdf
region: ap-northeast-1
imageId: ap-northeast-1/ami-eec1c380
minRam: 2000
loginUser: centos
installDevUrandom: true
allocatePTY: true
/////////////////////////////////////////////////////////////////////
Now, in the AMP User interface, go to the classic UI once again and go to collections. Click on the little black plus sign again and select ‘location’ instead of YAML. After that choose, ‘Cloud’ and then click the green ‘next’ button. After that, choose ‘YAML’ and copy/paste the location blueprint you created earlier into the text area. After that click ‘add to catalog’.
After that, it’s time to actually start your nodes…
IV. Launching your nodes
Before we begin, a quick word of warning…
With hyperledger-brooklyn, the state of your node is directly tied to your application. If you start an application, it will spin up a server. Once again, be very careful about this. Make sure you are prepared to spend some money because, on AWS, the node clusters will be deployed on about 4 large servers and 2 micros. This means that running it will take about 100 dollars per week and a half or so. Unless you are large institution we recommend only launching these nodes in short bursts as test servers or for research purposes.
Now…let’s get this cluster going.
In the Classic UI, select the Application tab. On the left side, click the small black plus sign. From here a menu will pop up with a collection of applications. Scroll down until you see the hyper ledger fabric applications.
One of the applications is a single cluster. It will deploy 5–7 nodes: Three Validating nodes, a CLI Node, a Membership node, a Sequence Node, and a Root Validating node. We will go into the subtleties of managing each one of these nodes next session.
Fill out the information. Don’t worry about the names too much, they are used for identifying your locations and nodes on the AMP user interface and won’t effect the runtime of the nodes. After pressing ‘deploy’ just sit and enjoy watching the boxes spin as the amp takes care of getting the nodes going.
Wait about 10 minutes and you should start seeing green dots appearing next to the names of your nodes on the left-hand side. Each node is deployed on a docker instance on the server run as normal hyperledger nodes.
The second option is a multicluster spread across multiple locations. It requires you to provide at least 3 locations of servers. This will deploy at least 9 server nodes which can get very costly. You set this up the exact same way you set up the nodes above except at the pop up asking for node names and so forth click the ‘add another location’ button and choose a new server location. Multi-locational multi-clusters take a long time to deploy so be patient, just wait a moment and you will see those wonderful little green beads.
Next up
This post is just the first of a many part tutorial/documentation on how to deploy, manage, and run a permissioned blockchain. Likewise, as the AMP and Brooklyn blockchains begin to evolve we will be adding addendums. The next section will cover how to manage, troubleshoot, and run chaincode on your new blockchain. We will also be considering all of our guides and documentation living documents so if you see a problem, send us a message and we will make changes.
Thank you for reading and we hope to see ya again!
