Let’s play with Snappy Ethereum
Introduction
Even though our work on the Snappy Ethereum Framework is not yet finished and uploaded to the Ubuntu store, enough work has been done on it that people can already play with it and provide valuable feedback.
Prerequisites
First and foremost you will need to grab and install Ubuntu core. I would recommend using a Raspberry Pi 2 if you have one lying around, since this is the platform on which we are currently testing. Alternatively, you can use a local KVM Virtual machine or a Vagrant image.
Once you have that setup, you will have to build our framework and sideload it onto your snappy device in lieu of downloading it from the Ubuntu Store. The suggested host development machine is Ubuntu 15.10, because it contains all the latest packages/tools that are required to build snaps.
If you have not read about the key concepts of Snappy Ubuntu applications please do so now and then proceed to setting up your Ubuntu development host where the framework will be built.
Finally, since we are going to be cross-compiling things for the arm architecture, you will want to follow this guide in order to add the ‘armhf’ architecture to dpkg and modify your apt-sources so that they contain ‘armhf’ too. This is not required for building the framework itself, but it will be needed later on for testing some of the Snaps that will be using the framework.
Building the Framework as a Snap
Now you are ready to go ahead and clone the Snappy Ethereum Framework.
user@ubuntu ~$: git clone https://github.com/slockit/snappy-ethereum-framework
user@ubuntu ~$: cd snappy-ethereum-frameworkWe are going to be using a tool called ‘snapcraft’ that reads a configuration file written in yaml and generates the desired snap package.
Understanding snapcraft.yaml
The latest snapcraft file for the framework can be found here. Let’s go through its contents and see what each of the sections mean.
The above is just some simple metadata and general information about our package and is pretty much self explanatory.
This is where we specify to snapcraft that this is not a simple application but a framework. The framework policy defines what capabilities the applications that use this framework will have. We will come back to this later.
To build a snap, snapcraft needs to know how to build its parts. There are various plugins for tools like make or scons. In our case we want to build geth, the Go ethereum client, for 2 specific architectures. Unfortunately at the moment snapcraft does not accept targets in the make plugin.
In order to do something that’s not officially supported by an existing snapcraft plugin you must create your own locally in a specific directory relative to the root yaml file. Ours is hosted here. We built a simple plugin that executes any given shell script. The script’s convention requires that its first argument be the destination directory where the resulting files are to be placed. Our team has opened a PR to contribute this generic plugin upstream.
Using the plugin is pretty simple. As the user, you only need to provide the script’s location and the destination directory for the resulting binary files.
The geth building script can be found here. It clones geth in the temporary staging area and builds it for the native (assuming x86) and armhf architectures. Finally it moves the produced binaries into the snapping directory.
We are also using a snapcraft plugin for copying desired files directly into the snapping area. The file to copy is the selector script for geth. It is based on the snappy magic launch script, and it selects the appropriate binary to execute depending on the detected architecture.
The selector script also determines the value for most of the arguments to geth, then executes it. The astute reader will notice that when copying the script to the staging area, we renamed it to geth. This is because this is the geth binary as far as the snap is concerned.
An application or framework can have multiple binaries and services. A service is a binary that will always run in the background as long as the framework is installed on the device.
In our case the framework has to run an Ethereum client, which for now is the Go-Ethereum client. The `caps` field stands for capabilities. Applications in Ubuntu Core run in quite a restricted sandbox for security reasons. These are the capabilities our service requires to run unhindered.
For more information on available capabilities and how confinement works in Snappy Ubuntu please refer to this page.
Building and Sideloading the Snap
To build the framework, all we have to do is type `snapcraft` in the same directory as the yaml file. Snapcraft will then go through all of the packaging stages until it produces a final package file called `ethereum_0.1_multi.snap`.
One additional package not mentioned in the official Ubuntu guide but handy for checking the final snap produced is click-reviewers-tools.
user@ubuntu ~$: sudo apt-get install click-reviewers-tools
user@ubuntu ~$: click-review ethereum_0.1_multi.snapThis will provide debugging and warning information about your snap and is a very good way to determine why a snap does not work or isn’t properly sideloaded.
Errors
------
- lint:snappy_type_redflag
(MANUAL REVIEW) type ‘framework’ not allowed
https://developer.ubuntu.com/en/snappy/guides/frameworks/ethereum_0.1_multi.snap: FAIL
In our case a red flag is raised as framework snaps need manual approval by Canonical. We can safely ignore this and proceed with sideloading.
user@ubuntu ~$: snappy-remote — url=ssh://ubuntu@DEVICEIP install ethereum_0.1_multi.snapAfter replacing the IP of your Snappy device in the command above, you will be able to sideload the framework. If successful, you should see an output similar to this:
install ethereum_0.1_multi.snap
=======================================================
Installing ethereum_0.1_multi.snap from local environment
Installing /tmp/ethereum_0.1_multi.snap
2016/01/24 13:29:29.928918 verify.go:85: Signature check failed, but installing anyway as requestedName Date Version Developer
ubuntu-core 2016–01–20 6 ubuntu
docker 2016–01–18 1.6.2.005 canonical
ethereum 2016–01–24 ILCgcTeVVTUC sideload
snappy-debug 2016–01–14 0.10 canonical
webdm 2016–01–18 0.11 canonical
pi2 2015–11–13 0.16 canonical
=======================================================
Notice the `sideload` as the Developer’s ID - the app is not from any official source (yet).
Exploring the Framework
Now it’s time to go into your Snappy device and see if your framework functions properly.
user@ubuntu ~$: ssh ubuntu@DEVICEIP
(RaspberryPi2)ubuntu@localhost:~$ sudo snappy listName Date Version Developer
ubuntu-core 2016–01–20 6 ubuntu
docker 2016–01–18 1.6.2.005 canonical
ethereum 2016–01–21 IKdUTOMILNOe sideload
snappy-debug 2016–01–14 0.10 canonical
webdm 2016–01–18 0.11 canonical
pi2 2015–11–13 0.16 canonical
And voila, you have your framework. Controlling applications in Snappy Ubuntu feels a lot like systemd. You can query the logs with:
(RaspberryPi2)ubuntu@localhost:~$ sudo snappy service logs ethereum2016–01–24T13:30:07.002172Z ubuntu-core-launcher I0124 13:30:06.998311 1367 server.go:555] Listening on [::]:30303
2016–01–24T13:30:07.006076Z ubuntu-core-launcher I0124 13:30:07.003039 1367 ipc.go:132] IPC service started (/var/lib/apps/ethereum/ILCgcTeVVTUC/.ethereum/geth.ipc)
2016–01–24T13:30:27.012972Z ubuntu-core-launcher I0124 13:30:27.010793 1367 downloader.go:288] Block synchronisation started
2016–01–24T13:34:32.765563Z ubuntu-core-launcher I0124 13:34:32.763499 1367 blockchain.go:1251] imported 2 block(s) (0 queued 0 ignored) including 0 txs in 4m4.629210919s. #2 [88e96d45 / b495a1d7]
You can start, stop and restart a service with:
sudo snappy service start ethereum
sudo snappy service stop ethereum
sudo snappy service restart ethereumTo understand the file system layout of Snappy apps and frameworks please read this. Our framework stores its data in the `$SNAP_APP_DATA_PATH` directory. Let’s explore it:
(RaspberryPi2)ubuntu@localhost:~$ ls -la /var/lib/apps/ethereum/total 16
drwxr-xr-x 4 root root 4096 Jan 24 13:29 .
drwxr-xr-x 6 root root 4096 Jan 21 23:27 ..
drwxr-xr-x 3 root root 4096 Jan 21 23:25 IKdUTOMILNOe
drwxr-xr-x 3 root root 4096 Jan 24 13:30 ILCgcTeVVTUC
lrwxrwxrwx 1 root root 12 Jan 24 13:29 current -> ILCgcTeVVTUC(RaspberryPi2)ubuntu@localhost:~$ ls -la /var/lib/apps/ethereum/current/total 12
drwxr-xr-x 3 root root 4096 Jan 24 13:30 .
drwxr-xr-x 4 root root 4096 Jan 24 13:29 ..
drwx — — — 5 root root 4096 Jan 24 13:30 .ethereum
Each version holds its data in a different directory. This enables the transactional updating nature of Ubuntu Core, allowing for quick and painless rollbacks in case of a mishap.
Using the Framework
The Ethereum framework provides the foundation required by Ethereum Đapps - saving them from having to run a full client each. How is this accomplished? I believe an example is in order.
Create an example Ethereum Snap
Head over to Github again and clone our example Snappy Đapp. There is a familiar snapcraft.yaml file in the root of the repository which we will use in order to build the example snap.
The snippet above is where the magic happens: our snap is set to depend on the Ethereum framework we just sideloaded in the previous section.
Our snappy app is composed mainly of two elements independent of one another. One is a a simple C binary to query the current block and the other is a node web server displaying the current block.
You will notice that we are using an awesome node.js plugin. It will clone the specified packages and their dependencies, build them and provide them to the final snap. We are using this plugin to get the Ethereum javascript library web3.js and some extra helper packages.
In the above snapcraft snippet is the description of the binaries we generate: The node web server and the test C application. Let’s look into their capabilities. As a server, the node-webserver should of course have the `networking` capability but what about `ethereum_ipc` that both binaries use?
It’s now time to look back to our framework and the framework policy directory. All frameworks come with such directory, which can contain a number of policy groups. A policy group is defined as a combination of a seccomp profile and an apparmor profile. Together these define a capability that the users of a framework can directly reference.
Here’s the apparmor profile for the IPC capability:
This gives the ability to potential applications using the framework to read and write from the IPC socket that geth creates.
The seccomp profile is a collection of system calls which the applications utilizing the IPC capability of the Ethereum framework are allowed to use. A link to the current IPC seccomp profile can be found here.
Build and Sideload the Example
What’s needed now is invoking snapcraft to build the example application. Once built, it will be sideloaded, like the framework.
user@ubuntu ~$: snapcraft
…
user@ubuntu ~$: snappy-remote — url=ssh://ubuntu@DEVICEIP install test_0.1_multi.snapSubstitute your own snappy device’s IP and you should be all ready to go. Note that if you use click-review on the generated snap you get the following:
Errors
------
- lint:framework
‘ethereum’ is not a supported framework
http://askubuntu.com/questions/460512/what-framework-should-i-use-in-my-manifest-file
test_0.1_multi.snap: FAILThe reason is that our framework is sideloaded and not yet an officially supported framework by Canonical. This will however not present any problem with our testing.
Using the Example Snap
Now it is time to use our example application! Login to your Snappy device and list all apps to make sure that the application is installed properly:
(RaspberryPi2)ubuntu@localhost:~$ sudo snappy listName Date Version Developer
ubuntu-core 2016–01–20 6 ubuntu
docker 2016–01–18 1.6.2.005 canonical
ethereum 2016–01–24 ILCgcTeVVTUC sideload
snappy-debug 2016–01–14 0.10 canonical
test 2016–01–24 ILDHAfWJVVIA sideload
webdm 2016–01–18 0.11 canonical
pi2 2015–11–13 0.16 canonical
If you see test in the output, then all is well. Let’s try to query our binary.
(RaspberryPi2)ubuntu@localhost:~$ sudo test.test-c-app
Result: {“id”:83,”jsonrpc”:”2.0",”result”:”0x1073b”}You might notice something strange here. It’s not enough to call the binary by name. It is namespaced, so you have to preface it with the application name. Snappy Ubuntu organizes its binaries this way to ensure proper separation between application installations.
The result is the raw JSON RPC response to the `getBlockNumber()` command. 0x1073b translates to block number 67387. Meaning that our device is still syncing.
And finally for something more interesting, let’s test the node web server that’s running as a service of our application. Navigate to your favorite browser, and go to http://DEVICEIP:8080/. Substitute “DEVICEIP’ for the IP of your Snappy device.
You should see the current block number being displayed and whenever you press refresh the number should be increasing, provided geth is syncing.
Conclusion
This was an introduction to Snappy Ubuntu, snapcraft and the Ethereum framework. Keep in mind that a lot of this — including Snappy Ubuntu itself — is under very active development and subject to change.
A lot remains to be done to complete the framework, including providing configuration ability via the official Snappy methodology, providing more concrete examples and fine-tuning the security policies so that Canonical can review and accept the framework in their store.
In the meantime, if any of you try this out and attempt creating/porting Đapps to Snappy Ubuntu, your feedback would be really appreciated. Either chat with us on our Slack or provide feedback directly in Github.
About the Author
Lefteris is the Technical Lead of slock.it
After graduating from the University of Tokyo, Lefteris has been developing backend software for various companies including Oracle and Acmepacket. He is an all-around tinkerer who loves to takes things apart and put them back together learning how they work in the process.
He has been part of Ethereum as a C++ core developer since November 2014, having worked on Solidity, the ethash algorithm, the core client and the CI system and is now leading the technical side of things towards revolutionizing the IoT world with the use of blockchains at Slock.it
Twitter: @lefterisjp
contact: lefteris@slock.it
