Tutorial: Build an open source smart city with LoRaWAN
City populations worldwide are growing at an unprecedented rate, which is resulting in more congested traffic, repairs, utility demand, etc. Many city officials and interest groups are leveraging technology to handle the steadily increasing load on buildings and infrastructure, all while increasing the quality of life for their citizens. For example, Seattle is using precipitation monitors to send flooding alerts, Barcelona is using internet-connected energy meters to reduce power consumption, and Amsterdam utilizes GPS apps to determine traffic flow and parking demand.
I saw quite a few potential use cases for smart cities/buildings in my neighborhood, so I decided to try building a prototype for a “smart city”. I chose to get started on building a system to monitor a nearby mid sized apartment building (~8000 sq. ft, 10 units, 2 stories). The building’s property manager had been plagued by a constant barrage of issues such as pests, utility malfunctions that went undetected for long periods of time, and security concerns like vandalism, illegal dumping, and cooking fires in the neighboring alley. The main goals here were to reduce overhead costs and make local residents feel safer.
To get started, I needed to determine a few things, such as which properties needed to be measured, how devices would communicate with one another, and where the devices would physically be located. For utility monitoring, I chose to get a few sound, humidity, vibration, and air quality sensors. For our security concerns, I selected a few flame, smoke, and PIR motion sensors. Originally, I wanted to use a few WiFi modules (ESP8266) to collect data from the sensors, but found that the maximum range of the building’s router wasn’t nearly enough to cover the perimeter (~20 meters). So, I looked into a few other IoT protocols such as Zigbee, Bluetooth, and LoRa.
I chose not to go with Zigbee or Bluetooth because they’re based on a mesh network topology, meaning that data is routed from one node to another, rather than each node sending data to a central gateway. Although it sounded like a very cool concept, I felt a mesh topology wouldn’t work as well in this case because many of the devices would be in “deep sleep” mode throughout the day in order to extend battery life.
Ultimately, I picked LoRaWAN for its advertised long range, low power consumption, and strong community support. Both the community forum and meetup group were instrumental in getting this setup up and running.
Hardware Requirements
Gateway:
- Raspberry Pi
- 915 MHz antenna
- Gateway Shield (Optional, but recommended)
- LoRA Gateway (RHF0M301)
Node:
- LoRA Feather
- 2500mAh Lithium Batteries
- Sensors (type depends on use case)
Gateway Setup
I started out by using the following diagram to connect the Raspberry Pi GPIO pins to the gateway using jumper wires.
As you can probably imagine, running wires directly from pin to pin like above got to be a bit tricky. Also, jumper wires can easily get unplugged when moving the gateway around. So I opted to get a gateway shield (bottom left) instead to end up with a much cleaner build.
To connect the LoRA gateway module to the Raspberry Pi , I had to get two sets of header pins and solder them into the section labeled “RAS PI”. Once that was done, I soldered the pins on the LoRA transceiver directly to the shields labeled “GATEWAY” section, and then plugged the Raspberry Pi’s GPIO pins into the headers.
After setting up the gateway, I continued on to create a free account at “The Things Network” console, and registered a device and an application. In this context, a device represents a physical gateway, and an application processes data published from associated devices.
To publish data to the registered TTN application, I pulled the recommended packet_forwarder project from github and ran its compile.sh script to generate the packet_forwarder binary. Finally, I placed the application credentials (id and key) in the /home/pi/.pktfwd.yml, and ran the following command to test the gateway connectivity
~/packet_forwarder/bin/packet-forwarder start --reset-pin 7 --config "/home/pi/.pktfwd.yml”
After seeing the successful connection message in the logs, I confirmed that I was able to see my gateway status in the TTN console
To ensure that the packet forwarder service would come back on after a reboot, I modified the systemd unit file provided in the git repository to include the reset-pin and config parameters in the command above, moved it to /etc/systemd/system/ttn-pkt-fwd.service, and ran systemctl enable ttn-pkt-fwd. If that works, running systemctl status -l ttn-pkt-fwd should show the following output.
Node Setup
After confirming that the gateway was working I moved forward to build a few “nodes”. In this context, the role of each node is to monitor connected sensors and publish the sampled data up to the gateway, which then streams the data to IoT / Cloud services for further processing.
This can be achieved by connecting a set of sensors to a LoRa radio module to a microcontroller such as an Arduino. I opted to get a few Adafruit LoRa feathers instead, as they seemed to be a pretty popular choice on the TTN forums, included USB and battery ports, and advertised a range of up to 20km.
Before wiring the node, I started out by installing a few software dependencies. You’ll need to install the Arduino IDE, and to follow the instructions here to add support for third party libraries.
I chose to use the arduino-lmic library, which is a modified version of the “LoRaWAN in C” package and optimized to run on Arduino and similar resource-constrained processors. This can be downloaded with
git clone https://github.com/matthijskooijman/arduino-lmic
After downloading the library, I adjusted the configuration file at arduino-lmic/src/lmic/config.h by changing the default frequency from 868 MHz (European standard) to 915MHz (American standard). I also enabled the LMIC_PRINTF_TO Serial value here to allow debugging output to be printed to the Arduino IDE’s serial monitor.
After that update, I let the Arduino IDE know where that modified arduino-lmic library was located by going to the menu “Sketch” -> “Include Library” -> “Add Zip Library”
Next, I moved on to edit the included example sketch in arduino-lmic/examples/ttn-abp/ttn-abp.ino . While running, this sketch repeatedly transmits “Hello World” message back to the gateway.
To get this working, I needed to log back into the console and obtain two sets of credentials; the Network Session Key and the App Session Key. These can be found by visiting the settings page at https://console.thethingsnetwork.org/applications/<application_name>/devices/<device_name>.
I copied those values from the console to update the NWSKEY and APPSKEY values on lines 38 and 43, which can be seen here.
Finally, update the values in the lmic_pins struct as seen below. This struct tells the LMIC library how to access the LoRa radio module.
const lmic_pinmap lmic_pins = {
.nss = 8,
.rxtx = LMIC_UNUSED_PIN,
.rst = 4,
.dio = {3,6,11} // dio0, dio1, dio2
};For this specific model of the LoRa feather, the .nss, .rst, and .dio0 pins are hardwired within the device, so those values should always be the same as what we have above. The second and third .dio pins can be changed if needed.
After updating the sketch, I continued on by soldering the included header pins so the feather could be plugged into a breadboard. Once that was done, I moved forward to make a few additional connections between the LoRa radio’s .dio1 / .dio2 pins and the microcontroller. This was done by connecting one wire from .dio1 to GPIO pin 6, and soldering a wire directly from .dio2 to GPIO pin 11. Finally, I soldered a wire to the antenna pin.
Once the node hardware setup was complete, I tested the connectivity by going back to the Arduino IDE, and clicking the “Upload” button (upper left icon next to the check mark).
Once/if this succeeds, go back to the TTN console and select Applications -> Application Name -> Data. If all the hardware connections are correct, a hex value equal to “Hello, World” should be showing up in the console.
Now that we’ve established connectivity between our node and gateway, we can start sending sensor data samples. To do this, we can plug a sensor into the breadboard, connect its negative/positive pins to the breadboard power rails, and its data pin to one of the feather “Analog input” pins (6 in total labeled like A0, A1, etc). In this example, I’ll use a sound sensor.
To test that the sensor was working properly, I adjusted the “loop” method to constantly print out the sensor value to Arduino’s serial monitor.
void loop() {
soundPin = A0;
soundValue = constrain(analogRead(soundPin), 0, 255);
Serial.println(F(soundValue));
}After verifying that the sensor was able to detect sound levels, the next step was to publish its values to the gateway, so they could be aggregated and analyzed. Each published packet can have a maximum size of 52 bytes (8 bits), and each byte can contain a value in the range of 0–255. ASCII strings take up a lot of bandwidth, (“Hello, World” was 26 bytes!), so it’s best to send a buffer of binary values instead of text or JSON objects.
Since this particular sound sensor has low precision unsigned values (constrained range from 0 to 255), we can fit each value in a single byte. To do this, we’ll adjust the do_send method on line 136 to initialize a byte array, run through a for loop and append sensor values to the array during each iteration. I set the sampling interval to measure the sensor level once every second, and append the average sound level to the byte array.
After flashing the updated sketch to the board, we can see a different set of incoming values in the TTN console, which takes the form of hex values.
Great, so now we’re able to publish sensor values at regular intervals to an internet-connected service. And to access the data without going through a web application, we can use a MQTT client to subscribe to the incoming values by using the following format
mqtt_sub -h us-west.thethings.network -p 1883 -u "${TTN_APPLICATION}" -t "${TTN_APPLICATION}/devices/${TTN_DEVICE}/up" -P "${TTN_ACCESS_KEY}"
This returns a json payload containing information for each packet. A few properties that appear to be especially useful are
hardware_serial— which node published the data payloadpayload— the contents of the data (our array of sound values in this case)timestamp— time the data packet was received
Now that we’re consistently receiving data from our node, we can move on to actually do something with the incoming data, such as streaming it to various analytics and visualization services. These services can be configured to reveal patterns, allow us to detect and respond to sudden shifts / anomalies, send alerts when certain values exceed a limit, and so on.
In this example, I’ll show how to visualize the incoming sound values on a time series plot in the Watson IoT platform. First, you’ll need to provision a Watson IoT service here, and a Cloudant NoSQL db here. Once those are up and running, go to the IoT dashboard, and enable the “Historian Data Storage” extension. This allows all incoming data received by the platform to be archived in the associated Cloudant database.
There are a few ways to forward data to the IoT platform. In this case, we’ll use MQTT to publish the sensor values. The steps to generate a set of MQTT credentials can be found here.
Before sending values, we’ll need to format them converting the byte array to a list of integers, and associating a timestamp to each value. We can get the approximate time the node began reading the sensor values by multiplying the sampling interval by the total number of samples taken, and subtracting that product from the time the message was received.
samplingInterval = 60000 // milliseconds
numSamples = 52// extract time from mqtt message, convert to UTC milliseconds
timeReceived = (new Date(ttnMessage.metadata.time)).getTime()startingTime = timeReceived - (samplingInterval * numSamples)
Now, we can simply loop through the array of values from our raw_payload attribute, and publish each time/value pair as a JSON object like so
// Publish time / sensor readings
for (i in _.range(0, values.length)) {
mqtt_client.publish(mqttTopic,
JSON.stringify({d: {
sound: values[i],
timestamp: new Date(startingTime + (samplingInterval * i))
}}))
}Running the snippet above will publish a set of values to the Watson IoT platform, where each message is in the format
{
"d": {
"sound": 83,
"timestamp": "2018-02-07T17:52:49.398Z"
}
}Once these values are all published, we can create a time-series visual, also referred to as a “board”. This can be created by going to the platform dashboard, clicking “Create New Board”
This will present a form asking for the event name and value(s) to be plotted
Once that form has been filled out, we can view the archived data by clicking the date, unchecking the “real-time” radio button, and selecting the date/time range. Since this data payload was published to TTN on 2/7/18, we can adjust the date range to look between the 6th to the 8th of February, and zoom in using the slider icons.
This article focused more on getting started with LoRaWan, “The Things Network”, and the Watson IoT platform. Please stay tuned for a few follow-up articles, where we’ll talk about challenges deploying these nodes in the field (power management, interference, clock drift). We’ll also talk about how to apply data science and machine learning algorithms to the aggregated datasets. These algorithms can be used to detect unusual activity and to find patterns and relationships between values.
Sources:
https://github.com/matthijskooijman/arduino-lmic#pin-mapping
TTN limitations, best practices
