IOTA cIRI on a BeagleBone Black with Yocto Project and OpenEmbedded
This is my first article for the IOTA community. It’s an introduction of my vision for the meta-iota OpenEmbedded layer, as well as a hands-on tutorial that acts as a proof-of-concept.
Introduction
The Yocto Project is an open source collaboration project that helps developers create custom Linux-based systems regardless of the hardware architecture. Meanwhile, OpenEmbedded is a build automation framework and cross-compile environment used to create Linux distributions for embedded devices.
Together, these projects provide a flexible set of tools and a space where embedded developers worldwide can share technologies, software stacks, configurations, and best practices that can be used to create tailored Linux images for embedded and IoT devices, or anywhere a customised Linux OS is needed.
There is an ever-growing movement of adoption of YP/OE tools inside the Embedded Linux community. This is mainly due to the flexibility and modularization they provide in terms of cross-compilation of different applications, customised kernels and Board Support Packages (BSPs). Bringing IOTA into YP/OE will potentially accelerate the adoption of IOTA by the Embedded Linux community since many different boards can now support IOTA tools.
The main tool used by YP/OE is called BitBake. Here is a short description borrowed from Wikipedia:
BitBake is a make-like build tool with the special focus of distributions and packages for embedded Linux cross compilation, although it is not limited to that. It is inspired by Portage,[3] which is the package management system used by the Gentoo Linux distribution. BitBake existed for some time in the OpenEmbedded project until it was separated out into a standalone, maintained, distribution-independent tool. BitBake is co-maintained by the Yocto Project and the OpenEmbedded project.
BitBake recipes specify how a particular package is built.[4] Recipes consist of the source URL (http, https, ftp, cvs, svn, git, local file system) of the package, dependencies and compile or install options. They also store the metadata for the package in standard variables.[5] During the build process, recipes are used to track dependencies, performing native or cross-compilation of the package and package it so that it is suitable for installation on the local or a target device. It is also possible to create complete images consisting of a root file system and kernel. As a first step in a cross-build setup, the framework will attempt to create a cross-compiler toolchain suited for the target platform.
It’s also good to know a little bit about Poky. Here’s a description borrowed from it’s official page:
Poky is a reference distribution of the Yocto Project®. It contains the OpenEmbedded Build System (BitBake and OpenEmbedded Core) as well as a set of metadata to get you started building your own distro. To use the Yocto Project tools, you can download Poky and use it to bootstrap your own distribution. Note that Poky does not contain binary files — it is a working example of how to build your own custom Linux distribution from source.
If you want more information, the Yocto official documentation is known for its quality and extent. Here are some useful links:
The first IOTA project I ported into YP/OE was cIRI. It is a low level implementation of an IOTA node written in C. It is meant to be a lightweight alternative version of the IOTA Reference Implementation (IRI), which is originally written in Java and not really well suited for Embedded Systems with RAM restrictions.
The project cIRI originally uses the Bazel build system, which already supports a lot of cross-compilation possibilities. At first, I was a bit scared about integrating Bazel into BitBake. However, I soon found out that Mr. Hongxu Jia has already done a great job of porting Bazel into YP/OE on his meta-tensorflow layer. Nevertheless, a few adaptations were introduced into the cIRI recipe so the Bazel workspace would integrate well with BitBake and also so that snapshots, sql and config files were populated into the final package.
The BeagleBone Black is based on the AM335x 1GHz ARM® Cortex-A8. I chose the BBB for this tutorial because Poky supports it out-of-the-box and it is quite a popular SBC on the YP/OE community. However, the layering system of BitBake should apply to any other boards supported in the YP/OE space. Check if there is any manufacturer documentation or guides about using Yocto for your board. If you need help applying this tutorial to other SBCs or SOMs, feel free to write a comment below, DM me on Discord (Bernardo [IF]#8478) or send me an email (bernardo.araujo@iota.org).
Fun fact: I was able to run cIRI inside QEMUlated MIPS and PowerPC machines! That shows how exotic meta-iota allows you to go 😉
Anyways, that intro is already too lenghty. Let’s get our hands dirty!
Tutorial
- Prepare your host for yocto development. Refer to Yocto’s official documentation for more details on setting up.
Assuming you are using Ubuntu (16.04 or 18.04):
$ sudo apt-get install gawk wget git-core diffstat unzip texinfo gcc-multilib build-essential chrpath socat cpio python python3 python3-pip python3-pexpect xz-utils debianutils iputils-ping python3-git python3-jinja2 libegl1-mesa libsdl1.2-dev xterm2. Clone poky. We are going to use the warrior branch of the Yocto Releases:
$ mkdir ${HOME}/yocto
$ cd ${HOME}/yocto
$ git clone git://git.yoctoproject.org/poky -b warrior3. Clone the other OE layers:
$ git clone https://github.com/openembedded/meta-openembedded -b warrior
$ git clone http://github.com/bernardoaraujor/meta-iota4. Start the build environment:
$ source poky/oe-init-build-env5. Inform BitBake about our layers by adding them to conf/bblayers.conf:
...
BBLAYERS ?= " \
${HOME}/yocto/poky/meta \
${HOME}/yocto/poky/meta-poky \
${HOME}/yocto/poky/meta-yocto-bsp \
${HOME}/yocto/meta-openembedded/meta-oe \
${HOME}/yocto/meta-iota/meta-tangle \
"6. Inform BitBake that we want to build for a BeagleBone and start building an iota-image-minimal image:
$ MACHINE="beaglebone-yocto" bitbake iota-image-minimalThis will probably take a while. BitBake fetches all the source code and builds all the different packages of the target image with a few parallel tasks. That’s why it can be a slow process. It’s still fun to watch though 🤓
7. After the build has finished, you can check the build artifacts:
$ ls ${HOME}/yocto/build/tmp/deploy/images/beaglebone-yocto/
am335x-bone--5.0.3+git0+31de88e51d_3df4aae607-r0-beaglebone-yocto-20190723214637.dtb
am335x-bone-beaglebone-yocto.dtb
am335x-boneblack--5.0.3+git0+31de88e51d_3df4aae607-r0-beaglebone-yocto-20190723214637.dtb
am335x-boneblack-beaglebone-yocto.dtb
am335x-boneblack.dtb
am335x-bone.dtb
am335x-bonegreen--5.0.3+git0+31de88e51d_3df4aae607-r0-beaglebone-yocto-20190723214637.dtb
am335x-bonegreen-beaglebone-yocto.dtb
am335x-bonegreen.dtb
iota-image-minimal-beaglebone-yocto-20190724173503.rootfs.jffs2
iota-image-minimal-beaglebone-yocto-20190724173503.rootfs.manifest
iota-image-minimal-beaglebone-yocto-20190724173503.rootfs.tar.bz2
iota-image-minimal-beaglebone-yocto-20190724173503.rootfs.wic
iota-image-minimal-beaglebone-yocto-20190724173503.rootfs.wic.bmap
iota-image-minimal-beaglebone-yocto-20190724173503.testdata.json
iota-image-minimal-beaglebone-yocto.jffs2
iota-image-minimal-beaglebone-yocto.manifest
iota-image-minimal-beaglebone-yocto.tar.bz2
iota-image-minimal-beaglebone-yocto.testdata.json
iota-image-minimal-beaglebone-yocto.wic
iota-image-minimal-beaglebone-yocto.wic.bmap
MLO
MLO-beaglebone-yocto
MLO-beaglebone-yocto-2019.01-r0
modules--5.0.3+git0+31de88e51d_3df4aae607-r0-beaglebone-yocto-20190723214637.tgz
modules-beaglebone-yocto.tgz
u-boot-beaglebone-yocto-2019.01-r0.img
u-boot-beaglebone-yocto.img
u-boot.img
zImage
zImage--5.0.3+git0+31de88e51d_3df4aae607-r0-beaglebone-yocto-20190723214637.bin
zImage-beaglebone-yocto.bin8. Insert the SD card on your desktop/laptop. In my case it showed up as /dev/sdc, might be something else for you. One way to find out is to use de lsblk command before and after inserting the SD card. Note that my SD shows up as /dev/sdc(along with my Ubuntu and Fedora HDs)
$ lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sdb 8:16 0 238,5G 0 disk
├─sdb2 8:18 0 230,1G 0 part /
├─sdb3 8:19 0 7,9G 0 part [SWAP]
└─sdb1 8:17 0 512M 0 part
sda 8:0 0 238,5G 0 disk
├─sda2 8:2 0 1G 0 part
├─sda3 8:3 0 237,3G 0 part
│ ├─fedora-home 253:1 0 179,5G 0 lvm
│ ├─fedora-root 253:2 0 50G 0 lvm
│ └─fedora-swap 253:0 0 7,8G 0 lvm
└─sda1 8:1 0 200M 0 part /boot/efi
$
$ lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sdb 8:16 0 238,5G 0 disk
├─sdb2 8:18 0 230,1G 0 part /
├─sdb3 8:19 0 7,9G 0 part [SWAP]
└─sdb1 8:17 0 512M 0 part
sdc 8:32 1 14,4G 0 disk
└─sdc1 8:33 1 14,4G 0 part
sda 8:0 0 238,5G 0 disk
├─sda2 8:2 0 1G 0 part
├─sda3 8:3 0 237,3G 0 part
│ ├─fedora-home 253:1 0 179,5G 0 lvm
│ ├─fedora-root 253:2 0 50G 0 lvm
│ └─fedora-swap 253:0 0 7,8G 0 lvm
└─sda1 8:1 0 200M 0 part /boot/efi9. Yocto provides a really useful tool called wic that packs up all the necessary build artifacts (MLO, U-Boot, kernel, rootfs) into a single .wic file that already contains the partitioning scheme for the SD card or eMMC. The BeagleBone Black’s BSP already gets BitBake to write a .wic image file. So all we need to do is use dd to flash the image file into the SD card.
$ cd ${HOME}/yocto/build/tmp/deploy/images/beaglebone-yocto/
$ sudo dd bs=4M if=iota-image-minimal-beaglebone-yocto.wic of=/dev/sdc
$ sync10. I assume you have a FTDI USB-RS232 cable, and that it shows up as /dev/ttyUSB0. Insert the SD card on the board, connect the FTDI and start minicom:
$ sudo minicom -D /dev/ttyUSB0 -b 11520011. Power up the board and stop uboot by pressing space. Type the following commands to load the kernel from the SD card:
# mmc dev 1
# mmc erase 0 512
# boot12. Now you are inside the BBB’s Linux! Log in with username root. Connect an Ethernet cable from your LAN router into the board. Find out the BBB’s local IP with the ifconfig command (192.168.1.35 in my case):
# ifconfig
eth0 Link encap:Ethernet HWaddr 78:A5:04:DC:8E:58
inet addr:192.168.1.35 Bcast:192.168.1.255 Mask:255.255.255.0
inet6 addr: fe80::7aa5:4ff:fedc:8e58/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:18 errors:0 dropped:0 overruns:0 frame:0
TX packets:11 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1924 (1.8 KiB) TX bytes:1470 (1.4 KiB)
Interrupt:45
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)13. Open a new terminal, and connect to the BBB via ssh:
$ ssh root@192.168.1.35
The authenticity of host '192.168.1.35 (192.168.1.35)' can't be established.
ECDSA key fingerprint is SHA256:CJXKu2BfUblmjpx4Pf0aF5ziBAXx62X8WfHA0/Q4/W8.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.1.35' (ECDSA) to the list of known hosts.
Last login: Wed Jun 19 17:48:00 201914. Now we are ready for cIRI. First, set up the database. Here, you can choose between mainnet.
# mkdir -p /etc/iota/db
# sqlite3 /etc/iota/db/spent-addresses-mainnet.db < /etc/iota/sql/spent-addresses-schema.sql
# sqlite3 /etc/iota/db/tangle-mainnet.db < /etc/iota/sql/tangle-schema.sql15. Edit the configuration file /etc/iota/config/conf.yml to add some neighbor IP/URL and port information. Just uncomment the following lines and fill the fields with the correct values for neighbors IP/URL and port, as well as your own local cIRI port. Also make sure that your router has the corresponding TCP and/or UDP ports properly configured on its Port Forwarding and Firewall configurations.
For example:
log-level: info
...
neighbors: "udp://x.x.x.x:15600 tcp://y.y.y.y:15600"
...
http-port: 14265A good place to find nodes is the #nodesharing channel on the IOTA Discord.
16. Download and unpack the latest mainnet snapshot:
# wget https://db.iota.partners/iri-mainnet-snapshot.tar.gz
# mkdir local_snapshots
# tar zxvf iri-mainnet-snapshot.tar.gz mainnet.snapshot.meta mainnet.snapshot.state -C local_snapshots/17. Run cIRI with the following:
# ciri \
--config /etc/iota/config/conf.yml \
--local-snapshots-enabled true \
--local-snapshots-base-dir=/home/root/local_snapshots \
--snapshot-signature-skip-validation true \
--local-snapshots-pruning-enabled true \
--tangle-db-path=/etc/iota/db/tangle-mainnet.db \
--spent-addresses-db-path /etc/iota/db/spent-addresses-mainnet.dbThere you go! You have a cIRI node running on your BBB!
Conclusion
That’s all folks. I hope this is useful for the IOTA Community.
Please let me know your thoughts on meta-iota!
p.s.: this tutorial is only meant to act as a proof of concept. The principles demonstrated here should not applied for production environments, where many other customisations to the final image are necessary in order to create a secure environment for an IOTA node.
