Mainflux Edge (part 2.)

This is the second article in series about running Mainflux on the edge, and in this article we will cover services running on the edge gateway: Agent and Export.

We at Mainflux Labs are building one of the world’s most sophisticated IoT system — Unified Cloud-Native IoT Platform. Open-sourced under Apache-2.0 license and published on our GitHub.

Mainflux Edge Gateway

In the previous article we have deployed Mainflux on Raspberry Pi thus enabling Raspberry Pi to manage IoT devices. In this article we will proceed further and deploy two additional services Agent and Export that will serve as a bridge between gateway and Mainflux cloud instance. Mainflux core services deployed together with Agent and export are main building components for edge gateway in Mainflux IoT platform.

Fig. 1- Mainflux Gateway Services

Deploying Agent and Export on Raspberry Pi

First lets deploy Mainflux on some machine that will be cloud host for Mainflux IoT Platform(DigitalOcean can be good candidate). Raspberry Pi will be connected via Agent and Export service with cloud. Raspberry Pi data and control will be available in cloud instance of Mainflux. We will need to start Mainflux core services along with bootstrap and provision service. We can start UI docker-compose which will deploy all necessary services.

git clone https://github.com/mainflux/ui
cd ui
nano docker/configs/config.toml

Edit config.toml for provision service, to enter proper host for MQTT both for Agent and Export service:

...
[bootstrap.content.agent.mqtt]
      mtls = false
      qos = 0
      retain = false
      skip_tls_ver = true
      url = "tcp://some-domain-name:1883"
...
 [bootstrap.content.export.mqtt]
      ca_path = "ca.crt"
      cert_path = "thing.crt"
      channel = ""
      host = "tcp://some-domain-name:1883"

some-domain-name can be IP address as well. Next, lets bring up the docker composition and start core Mainflux services along with addons and among them provision and bootstrap .

docker-compose -f docker/docker-compose.yml up

Provision service uses config.toml to create configuration that will be used for Agent and Export to start and connect to Mainflux cloud, so next step would be to provision a gateway using provision service /mapping endpoint, to access endpoint we will need, if not already created, to create user and obtain access token.

Next we will get the code for edge services and build executable that can be run on Raspberry Pi armhf architecture. Building can be done on your workstation or on Raspberry Pi ( but this will take a while), so we will do cross-build (I’m using Ubuntu 20.04 but, since services are written in Go, this should work on any platform).

Lest build Agent service:

cd $HOME
git clone https://github.com/mainflux/agent
cd agent
GOOS=linux GOARCH=arm GOARM=7 make
scp build/mainflux-agent pi@raspberrypi.local:

Next lets build Export service, and transfer it to Raspberry Pi

cd $HOME
git clone https://github.com/mainflux/export
cd export
GOOS=linux GOARCH=arm GOARM=7 make
scp build/mainflux-export pi@raspberrypi.local:

That is it, now lets switch to Raspberry Pi, we will copy service executables to $GOBIN so that it can be executed from shell.

ssh pi@raspberrypi.local
sudo cp mainflux-export /usr/local/go/bin/mainflux-export
sudo cp mainflux-agent /usr/local/go/bin/mainflux-agent

Next, we will create a script for starting Export service, create a file export_start with a following content:

#!/bin/bash 
MF_EXPORT_CONFIG_FILE=./export-config.toml mainflux-export

Then make script executable and copy it to executable path

chmod +x export_start
sudo cp export_start /usr/local/bin

Now we will start the the agent, using parameters that we supplied to /mapping endpoint in provision step external_id and external_key to obtain bootstrap configuration:

cd $HOME
mkdir gateway
cd gateway
MF_AGENT_BOOTSTRAP_SKIP_TLS=true \
MF_AGENT_BOOTSTRAP_ID=pi \
MF_AGENT_BOOTSTRAP_KEY=raspberry \
MF_AGENT_BOOTSTRAP_URL=http://some-domain-name:8202/things/bootstrap \
mainflux-agent

That is it, Agent is now connected

Error regarding saving export configuration can be ignored as we are going to send Export configuration via Agent control plane in the next step.

To configure and start export service we will use a configure.sh that I’ve made for this purpose, go back to Export repo, edit the configure.sh to setup parameters at the beginning of the script, these are the parameters that we used during provision step

MTLS=false
EXTERNAL_KEY='raspberry'
EXTERNAL_ID='pi'
MAINFLUX_HOST='mainflux.com'
MAINFLUX_USER_EMAIL='edge@email.com'
MAINFLUX_USER_PASSWORD='12345678'
EXPORT_CONFIG_FILE_TMPL='./configs/export-config.toml.tmpl'
EXPORT_CONFIG_FILE='./configs/export-config.toml'

Then just run the script which will create export-config.toml and send it to Raspberry Pi over Agent MQTT control channel and than start the execute script for Agent

scripts/configuration.sh 

You can check the contents of created config file in /configs

Now if you have followed previous article where we have deployed Mainflux on Raspberry Pi and you have it running on your Raspberry Pi device along with Agent and Export you can try publishing messages to gateway via MQTT control channel, also you can publish to NATS export topic, export service will pick them from NATS bus and send them to Mainflux cloud via MQTT on the channel specified in [[routes]].

Now you can subscribe to Agent response topic and send some commands to be executed on remote device or to view services that are sending heartbeat.

Also you can check terminal in the UI where you can execute bash commands on Raspberry Pi. Commands you enter in browser are sent via MQTT to Agent and response from remote shell is sent back via MQTT to browser.

If you look at the Fig. 1 you can see all the interconnections between services on the gateway. Agent and export are connected to the NATS bus, NATS bus can be deployed standalone or as a part of Mainflux composition.

Services 1..3 would be some custom developed services that are connecting to devices via BLE, ETH, WiFi, GPIO or any other interface that gateway supports, services should publish to NATS so that Export can send them to cloud. Export service would have route configured to listen for nats_topic = “export.service1” for service 1.

If you look at the config file that we have create for Export service there are two routes, one is for collecting messages from export.> and the other one is for collecting messages from channels.> which are actually messages from local running Mainflux on the gateway, all the messages that MQTT subscribed devices are publishing are going over NATS subject channels and Export will collect them and export to the Mainflux in cloud.
On the diagram you can see that Export service is sending a heartbeat to the NATS, Agent is subscribed to the heartbeat.> and keeps a track of services that are running.

I hope you’ll find this enough informative to get you started and to grasp how to use this services.

Until next time,
Mirko Teodorović

To learn more about Mainflux and how we’re building the world’s most sophisticated open-source Industrial IoT cloud, visit our website and follow us on Twitter!