ESP32 Devlog 11 — Data Visualization

Asynchronous Web Server, SPIFFS, and now what?

In the previous web server project, we made a simple ESP32 asynchronous web server with AJAX/XML that reads BMP180 sensor readings and display the result in the web browser and discovers ESP32 SPIFFS as a filesystem and utilize it to store HTML file. Here’s the previous link to the old projects related to this devlog:

• ESP32 Devlog 3 — Barometric Sensor
• ESP32 Devlog 9 — Real-Time Weather Station Web
• ESP32 Devlog 10 — SPIFFS Filesystem

In this project, i want to visualize the readings from the BMP180 sensor using Highcharts. I won’t explain nor show the physical ESP32 schematic diagram with BMP180 here. You can check the configuration on my previous ESP32 Devlog 3.

Highcharts is a modern SVG-based, multi-platform charting library. It makes it easy to add interactive charts to web and mobile projects. It has been in active development since 2009. You can find more of highcharts documentation here.

A. File Organization

I’m using the SPIFFS Filesystem to separate HTML, CSS, JS, and .ino files so the files does not crammed up when i’m working. I personally use SCSS as a CSS Preprocessor. I also worried about the file size i’m going to cram up to the ESP32 though. Here’s the directory tree:

Optionally, you can only upload style.css, script.js, and index.html for saving more data. The SCSS files are to be compiled file that can be ignored. If you happen to use VSCode as a text editor to your HTML, CSS, or JS file, you can use the live SASS compile from visual studio code to complie SASS/SCSS file to CSS file. Otherwise, just use the normal CSS.

B. Requirements and To-Do

These are the requirements for this project taken from Random Nerd Tutorial. Instead of BME280, i personally use BMP180 as a sensor.

Software Requirement

1. Arduino IDE with ESP32 Board Support
2. Visual Studio Code with live SASS compiler extension

Hardware Requirement

1. Breadboard
2. Jumper Cable
3. ESP32
4. BMP180

Service Requirement

1. ESP32 SPIFFS Filesystem Uploader tool
2. ESPAsyncWebServer library
3. AsyncWeb library
4. Adafruit BMP180 sensor library

Most of the links refer to the Random Nerd Tutorials, check the blog out for more reference and technical detail. This devlog is not a tutorial, rather it keeps me log the troubles i’ve got so far.

C. The Code

I am using the ESPAsyncWebServer Library for this project because i have some trouble in the HTTP GET routing. More on that later.

Arduino File

Replace ssid and password with your wireless network.

The .ino file is kinda straightforward. Though, you would like to visit the ESPAsyncWebServer documentation for more info. It was based from Random Nerd Tutorials about ESP32 Plot Sensor Readings.

How The Arduino Code Works

First, include the necessary libraries to the sketch.

#include <WiFi.h>
#include <Adafruit_BMP085.h>
#include "SPIFFS.h"
#include "ESPAsyncWebServer.h"

Then, create an instance to communicate with BMP180 Sensor using I2C:

Adafruit_BMP085 bmp;

Insert network credentials in the following variables:

const char* ssid      = ...;
const char* password  = ...;

Then, instantiate an Async Web Server object on port 80

AsyncWebServer server(80);

Create two functions to request the temperature and atmospheric pressure from the BMP180 sensor and return the readings as a String type

String readBMP180Temperature(){
  float t = bmp.readTemperature();
  return String(t);
}
String readBMP180Pressure(){
  float t = bmp.readPressure();
  return String(t);
}

In the setup(), initialize the serial baud rate, BMP180 sensor and SPIFFS filesystem

Serial.begin(115200);
// Hardware Checking
  if (!bmp.begin()) {
    Serial.println("Could not find a valid BMP085 sensor, check wiring!");
    while (1) {}
  }
if(!SPIFFS.begin(true)){
    Serial.println("An Error has occurred while mounting SPIFFS");
    return;
  }

Then, connect to WiFi and print the IP Address in the Serial Monitor

// Connect to WiFi
  Serial.print("Connecting to ");
  Serial.println(ssid);
  WiFi.begin(ssid, password);
  
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }
  
  Serial.println("\nWiFi Connected");
  Serial.println("IP Address: ");
// Local IP Address
  Serial.println(WiFi.localIP());

After that, there are routing functions which serves the GET request asynchronously.

// Routing
  server.on("/", HTTP_GET, [](AsyncWebServerRequest *request){
    request->send(SPIFFS, "/index.html");
  });
server.on("/temperature", HTTP_GET, [](AsyncWebServerRequest *request){
    request->send_P(200, "text/plain", readBMP180Temperature().c_str());
  });
server.on("/pressure", HTTP_GET, [](AsyncWebServerRequest *request){
    request->send_P(200, "text/plain", readBMP180Pressure().c_str());
  });
server.on("/style.css", HTTP_GET, [](AsyncWebServerRequest *request){
    request->send(SPIFFS, "/style.css", "text/css");
  });
server.on("/script.js", HTTP_GET, [](AsyncWebServerRequest *request){
    request->send(SPIFFS, "/script.js", "text/javascript");
  });

Finally, start the asynchronous web server. Because of this, the loop function remains blank.

server.begin();

The Innards of HTML and JS

HTML Code

JS file

The main concern of the HTML File is the javascript CDN for highcharts. The highcharts need to have an element with the same id as declared on JS file

<script src="https://code.highcharts.com/highcharts.js"></script>
<div id="chart-temperature" class="chart-visual"></div>
<div id="chart-pressure" class="chart-visual"></div>

For the JS file, You may make a Highcharts Chart object and fill the fields.

var chartT = new Highcharts.Chart({
  chart:{ renderTo : 'chart-temperature' },
  title: { text: 'BMP180 Temperature' },
  series: [{
    showInLegend: false,
    data: []
  }],
  plotOptions: {
    line: { animation: false,
      dataLabels: { enabled: true }
    },
    series: { color: '#059e8a' }
  },
  xAxis: { type: 'datetime',
    dateTimeLabelFormats: { second: '%H:%M:%S' }
  },
  yAxis: {
    title: { text: 'Temperature (Celsius)' }
  },
  credits: { enabled: false }
});
var chartP = new Highcharts.Chart({
  chart:{ renderTo:'chart-pressure' },
  title: { text: 'BMP180 Pressure' },
  series: [{
    showInLegend: false,
    data: []
  }],
  plotOptions: {
    line: { animation: false,
      dataLabels: { enabled: true }
    },
    series: { color: '#18009c' }
  },
  xAxis: {
    type: 'datetime',
    dateTimeLabelFormats: { second: '%H:%M:%S' }
  },
  yAxis: {
    title: { text: 'Pressure (hPa)' }
  },
  credits: { enabled: false }
});

Then make the setInterval function to create an AJAX request with set interval in miliseconds

setInterval(function ( ) {
  var xhttp = new XMLHttpRequest();
  xhttp.onreadystatechange = function() {
    if (this.readyState == 4 && this.status == 200) {
      var x = (new Date()).getTime(),
          y = parseFloat(this.responseText);
      //console.log(this.responseText);
      if(chartT.series[0].data.length > 10) {
        chartT.series[0].addPoint([x, y], true, true, true);
      } else {
        chartT.series[0].addPoint([x, y], true, false, true);
      }
    }
  };
  xhttp.open("GET", "/temperature", true);
  xhttp.send();
}, 1000 ) ;
setInterval(function () {
  var xhttp = new XMLHttpRequest();
  xhttp.onreadystatechange = function() {
    if (this.readyState == 4 && this.status == 200) {
      var x = (new Date()).getTime(),
          y = parseFloat(this.responseText);
      //console.log(this.responseText);
      if(chartP.series[0].data.length > 10) {
        chartP.series[0].addPoint([x, y], true, true, true);
      } else {
        chartP.series[0].addPoint([x, y], true, false, true);
      }
    }
  };
  xhttp.open("GET", "/pressure", true);
  xhttp.send();
}, 1000 ) ;

For the rest of the CSS code, you might want to visit my github repository because it will be too long if i explain each CSS blocks here.

D. The problem with Old Webserver Algorithm

You might notice that from the first time i began to tweak ESP32 web server, i only used the arduino WiFi library. I also use the SPIFFS as a non-volatile storage to embed my HTML file. The JS file and the CSS file are written within HTML file. The main concept is to utilize customized header from GET request with AJAX and to send the sensor readings to XML. It works when i only use a single html file because the .ino could fetch and write the html file to the client side. But, when i make another file on the tree, the browser just cannot read the JS and CSS file from my ESP32. It just read the index.html over again. I might use the bulit in routing strategy to utilize which file should be called when the browser send a GET request to the specified file. But, i think it’s too much and a hassle for me.

ESP Async Library To The Rescue

Yes, after a while reading the capabilities of ESPAsyncWebServer, it reduces the code significantly by letting the AsyncWebServer do the work at the begin block so the loop block could be used for other purpose.

Demonstration

E. Conclusion and Next Project

Data logging that is. This code snippet might be used for the next project. I highly recommend the Async Web Server for practical web server use. You may also use the traditional way for tweaking purpose.

It’s kinda hard to change the chart style though because every style injected by the script is embedded in the HTML inline tag. That makes the title unresponsive for mobile viewport.

The Mobile View