Using EdgeX as an IoT middleware
EdgeX is an open-sourced, vendor-neutral, edge IoT middleware platform that was previously developed by Dell and is now developed and maintained under the Linux Foundation. It is currently supported by Intel, Canonical, Dell, IBM, Wipro, and other major companies.
EdgeX translates and transforms sensor data from devices and delivers it to applications over network-based protocols that meet the needs of the clients. It is capable of sending as well as receiving data along with usage for the control and actuation of edge nodes.
As a part of my internship, I had worked with this platform and was surprised to see the lack of online support when faced with issues during development. I had to rely on the EdgeX Slack community to understand the cause of certain issues. This can be traced back to the reason that EdgeX is still not widely adopted for developing IoT solutions. The aim of this article is to share my experience along with a short guide on setting up a simple IoT use case.
GETTING STARTED
Now that we have a basic understanding of EdgeX, let’s move on to the development side of things. EdgeX consists of multiple microservices that are packaged in the form of docker container images maintained on DockerHub, which can be found here.
It is to be noted that this article addresses the scenario where EdgeX is deployed on an edge device such as the Raspberry Pi. Luckily, EdgeX supports arm64 as well as amd64 arch types. To distinguish between the container images for the two arch types, the images for arm64 end with ‘-arm64’. Hence, in this case, we shall be using the arm64 version of each microservice.
An example would be ‘edgexfoundry/docker-device-bacnet-c-arm64’ and its amd64 counterpart would be ‘edgexfoundry/docker-device-bacnet-c’
It is to be noted that EdgeX is under active development and has two major releases every year. The current release, as of the time of writing this article is the Hanoi release.
EdgeX can be run by either using the docker-compose file which can be found here or can be run using the snap store on Linux. From personal experience, I recommend going with the docker-compose method.
The docker-compose file contains the necessary version of each microservice to be pulled from DockerHub and to be used. It also permits the modification of certain parameters of the microservice which will be dealt with in the later section of this article. For future reference, the docker-compose file from here shall be used for the upcoming sections. It is to be noted that EdgeX stores the sensor data locally using Redis.
NOTE: The docker-compose file includes the bare minimum of microservices to start EdgeX and can be modified to include the necessary microservice as per need.
To run EdgeX on the Raspberry Pi, I recommend using the Ubuntu server found here and using SSH for the purpose of development. I shall not get into the explanation of the setup but will leave this link for reference. Once Ubuntu setup is ready with the necessary requirements and files downloaded, the following command on the terminal (in the directory of the docker-compose.yml file) shall get EdgeX up and running,
docker-compose up -dNOTE: You can skip the ‘-d’ if you would like to monitor the logs from each microservice
After EdgeX has started without any errors, we can view the status of each microservice by using the Consul service running on port 8500. Simply visit localhost:8500 on your web browser to open the GUI interface.
For the purpose of keeping the article concise, I shall not be going into every detail of a step, but shall include a reference that can be found here.
USE CASE
To get a better understanding of EdgeX, let’s take an example of temperature monitoring to demonstrate the functionality of EdgeX using MQTT protocol, InfluxDB, and Grafana.
In this simple case of temperature monitoring, a temperature sensor such as DHT11 can be used to take readings and actuation shall be done on the edge without the intervention of the user from the application layer for a quick response time. It is to be noted that the sensor is interfaced with an inexpensive Wi-Fi-capable microcontroller to transmit data via MQTT.
The microcontroller is configured to connect to an MQTT broker, set up in the VM environment on the cloud. For more information, this article can be used as a reference for setting up the sensor node. For debugging the MQTT messages, I recommend using MQTTLens as I feel it gets the job done. It is to be noted that the MQTT topic used to publish data will be used later, so make a note of it.
CONFIGURING EDGEX
As mentioned earlier, every EdgeX release contains only the essential microservices included. To make the development process easier, EdgeX has a console service that presents a GUI interface.
To include this service, paste the following code under services of the docker-compose file. It is to be noted that this service runs on port 4000 and can be viewed by visiting: localhost:4000 on your web browser.
services: // COPY THE CODE BELOW
ui:
container_name: edgex-ui-go
hostname: edgex-ui-go
image: edgexfoundry/docker-edgex-ui-go-arm64:1.3.0
networks:
edgex-network: null
ports:
- 127.0.0.1:4000:4000/tcp
read_only: trueNow to enable EdgeX to support MQTT communication as a part of its ‘device services’, we need to add the following code under ‘services’, to include the device-mqtt microservice.
device-mqtt:
container_name: edgex-device-mqtt
depends_on:
- consul
- data
- metadata
environment:
CLIENTS_COMMAND_HOST: edgex-core-command
CLIENTS_COREDATA_HOST: edgex-core-data
CLIENTS_DATA_HOST: edgex-core-data
CLIENTS_METADATA_HOST: edgex-core-metadata
CLIENTS_NOTIFICATIONS_HOST: edgex-support-notifications
CLIENTS_RULESENGINE_HOST: edgex-kuiper
CLIENTS_SCHEDULER_HOST: edgex-support-scheduler
DATABASES_PRIMARY_HOST: edgex-redis
EDGEX_SECURITY_SECRET_STORE: "false"
REGISTRY_HOST: edgex-core-consul
SERVICE_HOST: edgex-device-mqtt
hostname: edgex-device-mqtt
image: edgexfoundry/docker-device-mqtt-go-arm64:1.2.1
volumes:
- ./mqtt:/custom-config
networks:
edgex-network: {}
ports:
- 127.0.0.1:49982:49982/tcp
read_only: true
entrypoint:
- /device-mqtt
- --confdir=/custom-config
- --registry=consul://edgex-core-consul:8500A key point to note here is that, under entrypoint, we have a configuration directory for custom configurations i.e. on the same directory as the docker-compose file, we have a folder (I just named it as mqtt) that contains a configuration.toml. This file is used to configure the device-mqtt service to connect to the MQTT broker along with other configurations. It is to be noted that the topic under which data is published is to be configured here. The file can be found here. The key parameters to be modified would be under [Driver].
# Driver configs
[Driver]
...
IncomingHost = '<ip address of server running the MQTT broker>' IncomingTopic = '<Sensor data topic>'
IncomingUser = '<username for MQTT broker>'
IncomingPassword = '<password for MQTT broker>'
ResponseHost = '<ip address of server running the MQTT broker>'
ResponseUser = '<username for MQTT broker>'
ResponsePassword = '<password for MQTT broker>'
...Including this file ensures that every time the device-mqtt service starts, it automatically connects to the MQTT broker configured in the TOML file.
Before we start registering our ‘MQTT device’, there is one last microservice that is to be included. In the architecture diagram, we see that the sensor data is stored on the cloud database. To enable this, EdgeX has an export service called ‘app-service-mqtt’ where the locally stored sensor data is exported to the cloud database via MQTT messages. In this case, we shall be using InfluxDB to store real-time data for better visualization. It goes without saying, a database on InfluxDB is to be created for storing data. To include this microservice, add the following code under services.
app-service-mqtt:
container_name: edgex-app-service-configurable-mqtt
depends_on:
- consul
- data
environment:
BINDING_PUBLISHTOPIC: events
CLIENTS_COMMAND_HOST: edgex-core-command
CLIENTS_COREDATA_HOST: edgex-core-data
CLIENTS_DATA_HOST: edgex-core-data
CLIENTS_METADATA_HOST: edgex-core-metadata
CLIENTS_NOTIFICATIONS_HOST: edgex-support-notifications
CLIENTS_RULESENGINE_HOST: edgex-kuiper
CLIENTS_SCHEDULER_HOST: edgex-support-scheduler
DATABASES_PRIMARY_HOST: edgex-redis
EDGEX_PROFILE: mqtt-export
EDGEX_SECURITY_SECRET_STORE: "false"
MESSAGEBUS_SUBSCRIBEHOST_HOST: edgex-core-data
REGISTRY_HOST: edgex-core-consul
SERVICE_HOST: edgex-app-service-configurable-mqtt
SERVICE_PORT: 48101
WRITABLE_PIPELINE_FUNCTIONS_MQTTSEND_ADDRESSABLE_ADDRESS: <server ip address running MQTT broker>
WRITABLE_PIPELINE_FUNCTIONS_MQTTSEND_ADDRESSABLE_PORT: 1883
WRITABLE_PIPELINE_FUNCTIONS_MQTTSEND_ADDRESSABLE_PROTOCOL: tcp
WRITABLE_PIPELINE_FUNCTIONS_MQTTSEND_ADDRESSABLE_TOPIC: "<Topic name under which data is exported>"
WRITABLE_PIPELINE_FUNCTIONS_MQTTSEND_PARAMETERS_AUTORECONNECT: "true"
WRITABLE_PIPELINE_FUNCTIONS_MQTTSEND_PARAMETERS_RETAIN: "true"
WRITABLE_PIPELINE_FUNCTIONS_MQTTSEND_PARAMETERS_PERSISTONERROR: "false"
# WRITABLE_PIPELINE_FUNCTIONS_MQTTSEND_ADDRESSABLE_PUBLISHER:
WRITABLE_PIPELINE_FUNCTIONS_MQTTSEND_ADDRESSABLE_USER: "<MQTT broker username>"
WRITABLE_PIPELINE_FUNCTIONS_MQTTSEND_ADDRESSABLE_PASSWORD: "<MQTT broker password>"
hostname: edgex-app-service-configurable-mqtt
image: edgexfoundry/docker-app-service-configurable-arm64:1.1.0
networks:
edgex-network: {}
ports:
- 127.0.0.1:48101:48101/tcp
read_only: trueThe final step is to run a script to subscribe to the data published under the MQTT topic and store the data into the database. An example script can be found here.
Phewf! After all these steps, we can finally register our ‘MQTT device’.
To store any sensor data be it temperature, humidity, acceleration, and whatnot, EdgeX requires you to create ‘value descriptors’ which lets EdgeX know what kind of measured quantity it will deal with. Along with this, every sensor device has a ‘profile’ associated with it. The device profile is analogous to a class and the device is the object. A device profile contains the value descriptors along with core commands. A sample device profile can be found here. EdgeX requires the device profile to be registered, which can be done by simply adding a device profile using the UI microservice running on port 4000.
The last step is to add a device which can be done by clicking on the icon under ‘devices’ and using the ‘+’ button to add a device with the appropriate configurations.
With this setup, every time the sensor module publishes data, the data is fetched by EdgeX and exported to the cloud database. Additionally, to visualize this data, we can make use of Grafana, adding the InfluxDB database as a data source. To analyze the real-time status of the Raspberry Pi, we can make use of Telegraf to collect data and store it into another InfluxDB database. To give the finishing touch, we can use a Grafana template to visualize the data from Telegraf along with the sensor output. Since I didn’t have a temperature sensor with me, I generated random values for sensor data.
Now to add some value-based actuation in the mix, we shall be using the ‘Rules Engine’ microservice. EdgeX Foundry has collaborated with EMQ x Kuiper to integrate the rules engine into EdgeX.
First, we need to create a stream, which can be done via the UI service following which we can create a rule. It is to be noted that the syntax is similar to SQL. We can also configure the action to be performed when the condition has been met. This ranges from issuing a core command to the device to sending the result to an HTTP server. It is to be noted that the rule defined below takes values every 30 seconds.
END NOTE
Thus, in this article, we have seen a simple application of EdgeX to create an IoT solution. I hope that this article would be of some help to encourage developers to try EdgeX. If you would like me to cover a particular segment in EdgeX, do let me know in the comments section :)
