Reduce Your Energy Bills using MongoDB — Part 2 Getting the Temperature right

In Part 1 of this series, I built a system to monitor and display the energy coming into my house by recording the electric and gas meter readings every few minutes. In this post, I look at the largest single energy use: heating. I am recording what the temperature is in multiple rooms and determining if I am wasting energy by having rooms warmer than I need them to be for the time of day. This will let me adjust the boiler timings and radiator thermostats to get the optimal use of natural gas.

To do this I am using BBC micro:bits. These tiny, cheap, educational computers are designed to give school children an easy introduction to the basics of programming but are also an inexpensive way to get a display, buttons, sensors (including a thermometer), and wireless communications in a tidy little package. I happened to have half a dozen in a drawer from a previous educational project.

Configuring micro:bits to provide temperature readings via Bluetooth

BBC micro:bits have a built-in 2.4GHz radio. Normally, this is used to provide simple, direct communication between the devices themselves to teach about mesh networking, but you can choose instead to use a Bluetooth stack which lets them communicate with mobile phones and desktop computers.

You can program micro:bits with Python, Javascript, or using a GUI block-based language; I’m using the latter. I am also taking advantage of the fact that a Bluetooth temperature reporting service is an explicit part of the Bluetooth library for the micro:bit, so I have almost no coding to do.

To set up a micro:bit, you log into https://makecode.microbit.org/, Microsoft’s education platform. Select New Project and give it a name, and you are presented with a project with two empty event handlers, “on start” and “forever.”

Select Advanced->(+) Extensions and install the Bluetooth module. Then drag and drop blocks to create the required program — like this. The components are in the Bluetooth and Basic menus.

This enables the Bluetooth temperature service on program start. This is, strictly speaking, all the code you need, but I also decided to flash a check mark for one second on every connection to assist with any debugging, so I added that to the “on bluetooth connected” handler.

To write the compiled code to the micro:bit, plug in the micro:bit using a USB cable. It will appear as a new disk drive. Click Download in the GUI to get your compiled program and drag the downloaded file to the micro:bit device. That’s it — this environment was designed to be as easy to use as humanly possible, after all.

I’m powering them around the house using old usb chargers that have been building up in a drawer over the years as each phone or gadget came with one. I also have nice little black plastic cases for mine.

Pair Bluetooth devices with a Raspberry Pi

I’m using the same Raspberry Pi Zero to read the temperatures via Bluetooth as I used to read the electric and gas meters. First, I Bluetooth pair each micro:bit. Then, I have a Python script to poll them and record the readings every few minutes.

To pair a micro:bit, first put it in pairing mode by holding the two front buttons whilst you press and release the one on the back. It will show the Bluetooth logo and then a random pattern.

On the Raspberry Pi type

sudo bluetoothctl
scan on

When you see it say BBC micro:bit (name) XX:XX:XX:XX:XX:XX, take a note of the hex value part — that’s it’s address. Turn off scanning and pair the device with:

scan off
pair XX:XX:XX:XX:XX:XX

It should then confirm that it’s successfully paired.

Reading Bluetooth values from a device using Python

You can then install the bluepy library for Python and run the following code. (I use cron to run it every two minutes.)

sudo pip install bluepy

This scans for 10 seconds to see if it can find any devices with the name “micro:bit,” then for each, it connects and reads the value of the temperature characteristic from the temperature service. The UUID information denoting these characteristics, I found on the Lancaster University micro:bit site, but alternatively, it’s easy to scan for available services and characteristics from Python and infer the UUID from the characteristic and service names.

This script then maps the device address to a name and uses the MongoDB driver ‘pymongo’ to write this to MongoDB Atlas. I suspected the micro:bits were reading a little high, because they are taped to USB chargers, so I measured the temperature in the center of each room with a room thermometer and added a calibration value.

Getting the outside temperature from a weather service

I figured it would be good to have the outside temperature for reference too. It was snowing outside at the time and I realized that the outside temperature definitely had some influence on the energy used to heat the house. I did this by creating a scheduled trigger in Realm to call a free weather API (openweathermap). I added a trigger with the following code. For simplicity, I decided to just add the data to the same collection as the internal temperatures. This runs every 10 minutes inside the MongoDB Application Service back end.

Visualizing temperature data in MongoDB Charts

The first thing, once I had data, was to create a chart of temperature versus time for each location. This was as easy as selecting a continuous line chart as the type and dragging the temperature, date, and location fields over. This gave me this rather ugly graph. The problem here is that the micro:bit measures the temperature to the nearest degree, so we see step changes in temperature once it goes over a threshold.

I enabled line smoothing in the Charts Customize tab. but because of the frequency of our data points, this makes a barely perceptible change. It just rounds off the corners. Instead, I turned to $setWindowFields and added a pre-aggregation to the data to compute for each data point what the average value was of the point an hour either side of it — fairly extreme but also appropriate smoothing. To do this, I added the following code in the box above the graph (the pre-filter).

This then gave me a chart that looks more like this — much more readable and pleasing to the eye.

l also decided to add a limit to show only the last seven days of data. To do that, I extended the pre-aggregation to include a $match using $$NOW to get only seven days of data. The filter looks like this.

I then added a version of this chart to use discrete points rather than continuous, showing the hourly binning and 24 hour periodic to give an overview of how the time changes in a typical day. You can see when the heating comes on and goes off in the morning clearly from this chart.

Getting a table of current temperatures

The next thing I wanted was to see all the current temperatures. For that, I chose the TextTable chart type and then wrote a short pre-aggregation to get the latest temperature for each location. MongoDB 5.2 has a new $top operator to do this, but the highest version of MongoDB on the free tier I am using for this is 5.0, so I had to use $group instead. That isn’t a problem as long as you only want the highest/latest value for each group. Otherwise, you need $topN from 5.3. The aggregation I used was:

The output, once I turned on conditional formatting, looks like this.

Visualizing my room temperatures using heatmaps

I really wanted a graphical view of temperatures in my house (at least for downstairs, as we don’t bother heating the bedrooms normally). MongoDB Charts include a Choropleth chart type that lets you color US states or countries of the world according to some value field. This is limited to maps and area boundaries it already knows, though, and the inside of my house is, unsurprisingly, not an option.

On the other hand, Charts does have the ability to plot a heatmap, and a map of heat sounded exactly like what I was after. With a heatmap, the chart is an X/Y grid, and each cell of the grid is coloured according to some expression for records with those X and Y values.

The heatmap chart has a limit of 20,000 data points, which was enough for a 160x120 “pixel” image, so I wrote a Realm function to create 19,200 documents and treat them as pixels, allowing me to draw a fully custom bitmap chart. I even pulled in an 8-bit font from a ZX Spectrum to allow me to label them. That Realm function looks like this and is scheduled to run every 15 minutes using a trigger. The numbers at the start are the pixel bitmaps for the numbers in my font.

This then renders the following somewhat retro but ultimately informative chart which mirrors my house’s downstairs floor plan.

Baselining and adjusting central heating settings to save money

All this is pretty but it’s not really getting me to my goal of reducing heating costs. To reduce the bills, I need to understand where I am turning on the boiler too soon, or where a radiator thermostat is incorrectly adjusted. I have electronic radiator valves that let me select a time and temperature but I’m sure they are not set up well, and the boiler also has more timing flexibility than I’m using.

The first thing I needed to do was define what temperatures I wanted and when. We don’t really differentiate between the weekend and weekdays in this house when it comes to getting up and going to bed, so that made it easier. I created a collection which defined, for each room, what the desired temperature was at a given time. That has records that look like this. Notably, I only specify the end of the time period all time between that, and the previous record (or midnight) is assumed to be at that value.

I specify the ‘to’ time field as an integer Military/24hr time as it’s just easy to read 900 or 1800 and it makes it simple to work with.

To allow me to see what times I have configured, I used a text table type in charts:

I created a custom field calculated and then used that as dynamic column names, giving me this easy-to-read table. The definition for that custom field was as follows, putting Until in front of the number and adding a leading Zero if required to maintain sort order.

Charting the difference between desired and actual temperatures

The easiest way to figure out how to adjust the heating controls is to be able to see how and when the actual temperature is above or below the desired temperature. Creating that chart took a little more thought than the previous ones.

First, I took the previous smoothed temperature over time chart and added the following pre-aggregation stage, adding in a simple integer military time field computed from the data.

I then selected “lookup field” on the location fields to pull information from the target_temps collection into each record. I’ve already done this in the screenshot below, thus the error.

Then I added a virtual field “target” which works out what the target temperature is for the given time and how close I am to it. I filter all the values where “to” is less than “miltime,” then take the largest.

Now I can see for the last few days how close I was to target temperatures.

Finally, I added a filter by room on the dashboard and shared my dashboard with the world, which is why you can see how I’m getting on aligning my desired temperatures with my heating controls by clicking here. Or if you don’t want to see it live, it looks like this.

This exercise has been fun and informative for me. I’ve not done this much with Charts before. Next time, I’m going to tackle understanding what is using all my electricity.