3 key metrics everyone in IoT should care about
Or, “Why are IoT metrics even important?”
There are infinite benefits to connecting products for users, and operational benefits for product vendors. But taking a closer look at the reams of data produced by most connected products, we see it can be hard for vendors to distil the key metrics that are vital to business success.
I wanted to take a look at what these metrics might be for your connected-product business, and how to obtain them —
The SaaS (Software as a Service) community has produced a lot of good thinking about metrics, much of which we can borrow to analyse IoT solutions too — indeed one can think of an IoT solution as “HaaS” (Hardware as a Service).
Below are my suggestions for the key metrics which every IoT manager must have at their fingertips daily, based on our extensive engagement with IoT companies across many different verticals. They’re also based on the metrics that we use at DevicePilot. I make no apologies for being a bit evangelical about this because my reason for creating DevicePilot was my own first-hand experience that as you start to grow an IoT business, these are the numbers that can make the difference between garlanded success and ignominious failure — and until you measure them you’re flying blind.
1. % of devices “Up”
What is it and why does it matter?
The hardware in any IoT solution is really just a means of delivering a service, and it’s the service that your users are paying for (hopefully with some MRR!). So it’s vital to measure how well you’re actually delivering that service, and you have to drive up this quality-metric in order to scale. So if you only have 100 devices deployed, then a 5% problem is an annoyance, but if you have 100,000 devices deployed, then a 5% problem will kill your business stone dead.
How do you measure it?
The most fundamental metric of any connected device is probably just … is it connected?! Devices usually send back some kind of heartbeat just to say that they’ve been installed and they’re still alive, for example once an hour. Occasionally these pings will go missing for various reasons, but if you fail to see, for example, three in a row then there’s definitely a problem — with connectivity if nothing else — and you need to be the first to know.
But remember — you need to measure not just technical functionality, but also that the device is actually delivering service value. It might be online and healthy enough to send regular pings, but the application may have frozen (and so might be your users, if it’s e.g. a heating controller!). So you should find some metric which shows that the device is actually delivering utility, based on what it is. If it’s a heating controller, then at least create a metric to show that it’s connected, measuring temperature and not reporting any application errors.
And if your devices are delivered in clusters (e.g. ten ticket machines at a railway station) then a related metric you may also want to measure is “site availability” (e.g. if there are 10 machines on site, 1 of them is broken and 9 are in use, then there is currently no site availability when someone wants to buy a ticket — and this metric could even help you sell more ticket machines!).
2. Active users
What is it and why does it matter?
For many IoT systems, nirvana is that they be fully autonomous and not require any user interaction at all, getting on silently with making the world better. However for the time being in the real world, however wonderful and autonomous any IoT system is, ultimately it’s humans that decide its value. If no one is watching, no one is valuing your IoT solution. And in any system, including web and mobile apps, user engagement is the key leading indicator of “churn” — customers deciding to abandon your product.
How do you measure it?
This metric is best measured over a time-period which is long enough to smooth out natural cycles of human activity (i.e. at least a day) yet short enough to provide fast enough feedback about incipient churn (i.e. probably no more than a month). So you’ll hear companies talk about their “DAU, WAU or MAU” — the daily, weekly and monthly metric.
The two ways to measure user interaction are:
- Locally at the device, if it’s designed to have user interaction — simply using the device, however that is done.
- Remotely via e.g. phone app — are users logging in to check things? Have notifications been set up to send emails? Is just one person watching each account (in which case you’re vulnerable to them e.g. moving jobs) or is there a broader set of people engaged (in which case not only is more value being delivered, but also this account is probably “stickier”).
Some devices (for example, connected street lights) don’t really have local users, so in that case you’ll have to make do with remote users (like council employees checking on status, or being notified about outages). Other devices (e.g. a personal appliance) might just have one user, in which case this metric is “1”.
Yet other types of device (e.g. EV charging posts) might have multiple local users (the people doing the charging) and remote users (the operations and customer support teams, product managers etc.). In this case you may not be able to identify unique users, but you can at least say that e.g. charging point X has served 15 customers today.
With DevicePilot you can not only measure user engagement over time, but you can see typical patterns of activity over daily or weekly cycles (e.g. plot the last 30 days wrapped onto a single typical day). Understanding exactly when users interact with your product may give you useful insights into how it fits into the pattern of their daily lives (e.g. just before they go to work, over their lunch break etc.)
3. Your value to the world
Once you can measure the previous two metrics, you can combine them to produce a metric which all companies want to maximise — the amount of value you’re delivering to the world. In many cases this is as simple as multiplying the first two metrics together (“devices up” x “active users”) to measure the total value you’re delivering. Or you could measure “number of device-uses per day” (especially if you charge by the use). Or even convert that into units specific to your business, such as kWh if you’re an energy company.
What are your numbers?
If you’re interested in measuring for yourself, I’d recommend DevicePilot —
- With just a few clicks you can get your live device data flowing into DevicePilot.
- Then in the DevicePilot user interface:
- Use the View page to create a filter called “Up” defined as “Last seen less than 3 hours ago” (or whatever length is 3x your device heartbeat interval)
- Use the Cohort page to create a KPI to measure “percentage of time where Up”
- Use the Dashboard page to add a KPI widget to plot that KPI over the past 30 days, and set your traffic-light red/amber/green levels appropriately.
If you’d like more information on tracking IoT metrics or about DevicePilot, you can book a quick call here.
