How to build an IIoT system using Apache NiFi, MiNiFi, C2 Server, MQTT and Raspberry Pi

Photo by Brent De Ranter on Unsplash
  • Collect data from sensors to a gateway at every factory
  • Move sensors data from one or several factories to the Cloud or the Data Center
  • Automatically Warm-Deploy new configurations to all the edge devices
  • Support large scale data volume and end-to-end security

Industrial IoT architecture

There are plenty of IoT reference architectures. Often, in industrial settings, you don’t have direct access to sensors and control systems. A gateway is used to bridge the OT and the IT world. For this reason, IIoT architecture often includes edge devices, gateways, regional hubs and finally storage/processing systems.

IIoT architecture

System implementation

Let’s start building our prototype.

Preparing the Raspberry Pi: MQTT and MiNiFi

To install Mosquitto MQTT broker and MiNiFi agent, run the following commands on your Raspberry Pi.

sudo apt-get update
#install and run Mosquitto broker on default port 1883
sudo apt-get install mosquitto
#install and prepare MiNiFi agent
tar -xvf minifi-0.4.0-bin.tar.gz
cd minifi-0.4.0
#add mqtt processor
wget -P ./lib/
  • FileChangeIngestor
  • RestChangeIngestor
  • PullHttpChangeIngestor
nifi.minifi.notifier.ingestors=org.apache.nifi.minifi.bootstrap.configuration.ingestors.PullHttpChangeIngestor# Hostname on which to pull configurations fromnifi.minifi.notifier.ingestors.pull.http.hostname=c2-server# Port on which to pull configurations fromnifi.minifi.notifier.ingestors.pull.http.port=10080# Path to pull configurations fromnifi.minifi.notifier.ingestors.pull.http.path=/c2/config# Query string to pull configurations withnifi.minifi.notifier.ingestors.pull.http.query=class=iot-minifi-raspberry-agent# Period on which to pull configurations from, defaults to 5 minutes if commented

Installing and configuring the MiNiFi C2 Server

Install MiNiFi C2 server on a public server that’s reachable from the MiNiFi agents. You can use hierarchical C2 deployment for network constrained applications as described a few lines below. Run the following command to install the C2 server:

tar -xvf minifi-c2-0.4.0-bin.tar.gz
cd minifi-c2-0.4.0
  • The CacheConfigurationProvider, which looks at the directory on the filesystem or on S3
  • The DelegatingConfigurationProvider, which delegates to another C2 server to allow for hierarchical C2 structures
  • The NiFiRestConfigurationProvider, which pulls templates from a NiFi instance over its REST API

Installing and configuring the NiFi Server

Install NiFi on a server reachable from the C2 server and run it.

tar -xvf nifi-1.6.0-bin.tar.gz
cd nifi-1.6.0
./bin/ start

Deploy and start the application

Before starting the MiNiFi agents on the Raspberry Pi, let’s see if the C2 server is well configured. Open the following URL in your web browser : http://c2-server:10080/c2/config?class=iot-minifi-raspberry-agent&version=1 . The C2 Server replies with a file containing the configuration of the template we built, in YML format . That’s great.

Results of C2 Rest API call
C2 server logs after the Rest API call
./bin/ start
C2 Server logs
MQTT Logs after MiNiFi agent start

Automagic Warm Redeploy

Now that our IIoT is running and data is flowing from every factory to our data center, let’s deploy a new application. For our test, we will make a minor modification to our MiNiFi agent configuration. Go to the NiFi web UI and edit the updateAttribute processor. Set the “version” attribute to 2 instead of 1 and save the flow in a new template “iot-minifi-raspberry-agent.v2”. That’s all! The new application will be automagically deployed.

C2 Server reaction to a new template


Apache NiFi and its eco-system (MiNiFi and C2 server) are powerful tools for end-to-end IoT data management. It can be used to easily and quickly build advanced IoT applications with flexible architectures and advanced features (automatic warm deploy, data provenance, backpressure, and so on).

Claps and subscription



We’ve moved to and publish tons of tutorials each week. See you there.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store