ESP32, Data Visualization, and Data Logging
Hi and welcome to my last Embedded System article :) This article is very closely related to my previous article — Database Integration — so you might wanna check out that article first before jumping into this project.
Data Visualization
This time, we’ll try to visualize BMP280 sensor reading measurements that we have gathered from previous project. Now, why is data visualization important? Visual representations of data make it easier for people to understand complex information as well as grasp key messages and insights from it quickly and easily.
The image above provides a big picture of how our project works.
So, the first thing we should do is create a PHP file named esp-chart.php as a script to visualize our data.
Copy the code that I provided below and paste it onto the php script file. This script will connect to a MySQL database that stored BMP280 sensor readings, retrieves the 30 most recent readings, formats it to JSON file for use by Highcharts. Highcharts will then creates two charts, each will visualize either temperature or pressure, and then displays both chart on our web page. Do remember to change dbname, username, and password to match your database setup.
<!--
Rui Santos
Complete project details at https://RandomNerdTutorials.com
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files.
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
-->
<?php
$servername = "localhost";
// REPLACE with your Database name
$dbname = "REPLACE_WITH_YOUR_DATABASE_NAME";
// REPLACE with Database user
$username = "REPLACE_WITH_YOUR_USERNAME";
// REPLACE with Database user password
$password = "REPLACE_WITH_YOUR_PASSWORD";
// Create connection
$conn = new mysqli($servername, $username, $password, $dbname);
// Check connection
if ($conn->connect_error) {
die("Connection failed: " . $conn->connect_error);
}
$sql = "SELECT id, temperature, pressure, reading_time FROM SensorMeasurements order by reading_time desc limit 30";
$result = $conn->query($sql);
while ($data = $result->fetch_assoc()){
$sensor_data[] = $data;
}
$readings_time = array_column($sensor_data, 'reading_time');
// ******* Uncomment to convert readings time array to your timezone ********
/*$i = 0;
foreach ($readings_time as $reading){
// Uncomment to set timezone to - 1 hour (you can change 1 to any number)
$readings_time[$i] = date("Y-m-d H:i:s", strtotime("$reading - 1 hours"));
// Uncomment to set timezone to + 4 hours (you can change 4 to any number)
//$readings_time[$i] = date("Y-m-d H:i:s", strtotime("$reading + 4 hours"));
$i += 1;
}*/
$temperature = json_encode(array_reverse(array_column($sensor_data, 'temperature')), JSON_NUMERIC_CHECK);
$pressure = json_encode(array_reverse(array_column($sensor_data, 'pressure')), JSON_NUMERIC_CHECK);
$reading_time = json_encode(array_reverse($readings_time), JSON_NUMERIC_CHECK);
/*echo $temperature;
echo $pressure;
echo $reading_time;*/
$result->free();
$conn->close();
?>
<!DOCTYPE html>
<html>
<meta name="viewport" content="width=device-width, initial-scale=1">
<script src="https://code.highcharts.com/highcharts.js"></script>
<style>
body {
min-width: 310px;
max-width: 1280px;
height: 500px;
margin: 0 auto;
}
h2 {
font-family: Arial;
font-size: 2.5rem;
text-align: center;
}
</style>
<body>
<h2>ESP Weather Station</h2>
<div id="chart-temperature" class="container"></div>
<div id="chart-pressure" class="container"></div>
<script>
var temperature = <?php echo $temperature; ?>;
var pressure = <?php echo $pressure; ?>;
var reading_time = <?php echo $reading_time; ?>;
var chartT = new Highcharts.Chart({
chart:{ renderTo : 'chart-temperature' },
title: { text: 'BMP280 Temperature' },
series: [{
showInLegend: false,
data: temperature
}],
plotOptions: {
line: { animation: false,
dataLabels: { enabled: true }
},
series: { color: '#059e8a' }
},
xAxis: {
type: 'datetime',
categories: reading_time
},
yAxis: {
title: { text: 'Temperature (Celsius)' }
//title: { text: 'Temperature (Fahrenheit)' }
},
credits: { enabled: false }
});
var chartP = new Highcharts.Chart({
chart:{ renderTo:'chart-pressure' },
title: { text: 'BMP280 Pressure' },
series: [{
showInLegend: false,
data: pressure
}],
plotOptions: {
line: { animation: false,
dataLabels: { enabled: true }
},
series: { color: '#18009c' }
},
xAxis: {
type: 'datetime',
categories: reading_time
},
yAxis: {
title: { text: 'Pressure (hPa)' }
},
credits: { enabled: false }
});
</script>
</body>
</html>Go to our web dashboard then open Tools -> File Manager -> Upload Files and you will be redirected to 000webhost File Manager then upload esp-chart.php file to html_public folder.
Go over to https://yourdomain.000webhostapp.com/esp-chart.php and you shall see two charts visualizing sensor readings
Data Logging
Our next and last topic is Data Logging. Data logging is the process of recording data from sensors, instruments, or other sources over time and store it in digital or analog format and can be used for a variety of purposes, including analysis, monitoring, and control.
In this project, we’ll try to log temperature and pressure readings from BMP280 sensor to a Google Sheets using IFTTT (If This Then That) Webhook Service
The image above shows us the cycle of data logging for our project, except that we’re not using BME280 but BMP280 and our deep sleep duration will be 5 minutes.
ESP32 connects to WiFi then measure the temperature, humidity, and pressure of its surroundings. ESP32 will then communicate with IFTTT Webhook Service to publish its readings to Google Sheets then go to a deep sleep for 5 minutes. It will then wake up, connects to WiFi and the whole process repeats.
IFTTT Webhook Service Setup
IFTTT is a platform that allows us to integrate and automate processes across services and devices or also known as “applets”. Applets are automations that connect two or more services to create a new experience. Each service has unique triggers, queries, and actions that allow us to build different Applets.
Making your own Applet is simple. Start with the trigger, which is the If This part of the Applet. Next, select your action, the “Then That” portion of your Applet, which will happen when the Applet is triggered.
Now, let’s go over to their website and create an account.
After creating an account and choosing personalized preferences, go to their Explore page and click on “Create” button on the top left corner.
You should see this page below
Click the “Add” button then scroll over and choose Webhooks.
Once you’re redirected to trigger option page, choose “Receive a Web Request” trigger.
Click “Connect”
Type in arbitrary event name for our trigger
Next up, let’s modify the “Then This” part by clicking “Add” button
Scroll down and choose Google Sheets and then choose “Add row to spreadsheet” as our action
IFTTT will then request to connect with Google Sheets. Click on Allow.
Complete actions field by typing in all needed information as follows and then click “Create Actions”:
Click “Continue”, review our Applet then click “Finish”
Okay, we have successfully created our new applets
Visit Webhook Service page -> Documentation to attain your API key. Keep in mind to keep this key confidential.
ESP32 Assembly
After finishing all IFTTT setup, let’s move on to making the ESP32 assembly
Tools
- ESP32
- Breadboard
- BMP280. Make sure it is soldered
- Male-to-male jumper cables
- Micro-to-USB data cables
Preparation
If you haven’t set up Arduino IDE on your laptop/PC, check out this link
Illustration
Steps
- Plug ESP32 into the breadboard. You can pretty much put it anywhere you like but for the sake of convenience, it’s better if you plug it in the edge of the breadboard.
- Plug BMP280 into the breadboard.
- Connect 3v3 pins of the breadboard with VCC pins of BMP280.
- Connect GND pins of the breadboard with GND pins of BMP280.
- Connect D22 (GPIO22) with SCL
- Connect D21 (GPIO21) with SDA.
Source Code
Copy the code below and paste it onto Arduino IDE. Make sure to change ssid, password, and resource variable to match with your laptop internet connection and resource variable with your trigger and API key.
/*
* Rui Santos
* Complete Project Details https://randomnerdtutorials.com
*/
#include <WiFi.h>
#include <Wire.h>
#include <Adafruit_Sensor.h>
#include <Adafruit_BMP280.h>
// Replace with your network credentials
const char* ssid = "REPLACE_WITH_YOUR_SSID";
const char* password = "REPLACE_WITH_YOUR_PASSWORD";
// Replace with your unique IFTTT URL resource
const char* resource = "REPLACE_WITH_YOUR_IFTTT_URL_RESOURCE";
// How your resource variable should look like, but with your own API KEY (that API KEY below is just an example):
//const char* resource = "/trigger/bme280_readings/with/key/nAZjOphL3d-ZO4N3k64-1A7gTlNSrxMJdmqy3";
// Maker Webhooks IFTTT
const char* server = "maker.ifttt.com";
// Time to sleep
uint64_t uS_TO_S_FACTOR = 1000000; // Conversion factor for micro seconds to seconds
// sleep for 10 minutes = 600 seconds
uint64_t TIME_TO_SLEEP = 600;
// Uncomment to use BMP280 SPI
/*#include <SPI.h>
#define BMP_SCK 18
#define BMP_MISO 19
#define BMP_MOSI 23
#define BMP_CS 5*/
#define SEALEVELPRESSURE_HPA (1013.25)
Adafruit_BMP280 bmp; // I2C
//Adafruit_BMP280 bmp(BMP_CS); // hardware SPI
//Adafruit_BMP280 bmp(BMP_CS, BMP_MOSI, BMP_MISO, BMP_SCK); // software SPI
void setup() {
Serial.begin(115200);
delay(2000);
// initialize BMP280 sensor
bool status = bmp.begin(0x76);
if (!status) {
Serial.println("Could not find a valid BMP280 sensor, check wiring or change I2C address!");
while (1);
}
initWifi();
makeIFTTTRequest();
// enable timer deep sleep
esp_sleep_enable_timer_wakeup(TIME_TO_SLEEP * uS_TO_S_FACTOR);
Serial.println("Going to sleep now");
// start deep sleep for 3600 seconds (60 minutes)
esp_deep_sleep_start();
}
void loop() {
// sleeping so wont get here
}
// Establish a Wi-Fi connection with your router
void initWifi() {
Serial.print("Connecting to: ");
Serial.print(ssid);
WiFi.begin(ssid, password);
int timeout = 10 * 4; // 10 seconds
while(WiFi.status() != WL_CONNECTED && (timeout-- > 0)) {
delay(250);
Serial.print(".");
}
Serial.println("");
if(WiFi.status() != WL_CONNECTED) {
Serial.println("Failed to connect, going back to sleep");
}
Serial.print("WiFi connected in: ");
Serial.print(millis());
Serial.print(", IP address: ");
Serial.println(WiFi.localIP());
}
// Make an HTTP request to the IFTTT web service
void makeIFTTTRequest() {
Serial.print("Connecting to ");
Serial.print(server);
WiFiClient client;
int retries = 5;
while(!!!client.connect(server, 80) && (retries-- > 0)) {
Serial.print(".");
}
Serial.println();
if(!!!client.connected()) {
Serial.println("Failed to connect...");
}
Serial.print("Request resource: ");
Serial.println(resource);
// Temperature in Celsius
String jsonObject = String("{\"value1\":\"") + String(bmp.readTemperature()) + "\",\"value2\":\"" + String(bmp.readPressure()/100.0F)
+ "\",\"value3\":\"" + String(bmp.readAltitude(SEALEVELPRESSURE_HPA)) + "\"}";
// Comment the previous line and uncomment the next line to publish temperature readings in Fahrenheit
/*String jsonObject = String("{\"value1\":\"") + String(1.8 * bmp.readTemperature() + 32) + "\",\"value2\":\""
+ String(bmp.readPressure()/100.0F) + "\",\"value3\":\"" + String(bmp.readAltitude(SEALEVELPRESSURE_HPA)) + "\"}";*/
client.println(String("POST ") + resource + " HTTP/1.1");
client.println(String("Host: ") + server);
client.println("Connection: close\r\nContent-Type: application/json");
client.print("Content-Length: ");
client.println(jsonObject.length());
client.println();
client.println(jsonObject);
int timeout = 5 * 10; // 5 seconds
while(!!!client.available() && (timeout-- > 0)){
delay(100);
}
if(!!!client.available()) {
Serial.println("No response...");
}
while(client.available()){
Serial.write(client.read());
}
Serial.println("\nclosing connection");
client.stop();
}Take a look at code snippet below where I set sleep time to be 10 minutes
// sleep for 10 minutes = 600 seconds
uint64_t TIME_TO_SLEEP = 600;Testing
- Connect ESP32 with Laptop or PC using micro-to-USB data cable and click on Select Board then choose DOIT ESP32 DEVKIT V1.
- Click on the verify button and wait until a message “Done Compiling” pops up.
- Once you see “Connecting……” message, press and hold Boot button on ESP32 and release upon seeing message “Hard resetting via RTS pin…”. This is basically telling you that the upload process is successful.
- Press “EN” button on ESP32
Documentation
After uploading the code, open up Serial Monitor. You shall see message like below that indicates successful HTTP request
Each request is made with an interval of 10 minutes. Open Google Sheets BMP280_readings and you shall see new data measurements are being logged every 10 mins.
Closing
Yayyy! Finally we have learned how to do data logging with ESP32. As I said before, this is my last project of Embedded System. It really is fun exploring the world of IoT and I can’t wait to learn even more. Thank you so much for reading my blogs and byeeee!
