Tell me more Internet of Things — Part 4— IOTA cost and value comparison
IOTA— How to stream your device data via IOTA (MAM) messages to the distributed immutable Tangle and visualize its content.
Part 1 | Part 2 | Part 3 | Part 4| Part 5 |Part 6 | … | Part n
In Part 1 we introduced the targets and hardware in Part 2 Google IoT and Part 3 MindSphere.
This article compares IOTA and uses its protocols for our specific business / use case and compares it to the selected IoT platform solutions.
I send measurement data to the Tangle (IOTA’s immutable database network) and explore how to visualize it for our scaled, hopefully successful environmental sensor business case. The questions on how expensive will be this implementation per device and what is in for the user. Bearing in mind that the current network and protocol is under constant development and that the future importance and advantages of IOTA maybe only fully unfold when moving from Internet based to mesh IoT network based architectures. Furthermore for our use-case in my opinion a combination of cloud based and IOTA based features would be more appropriate for the moment anyways.
When it comes to Blockchain or in this case Distributed Ledger Technologies (DLT) usually its cryptocurrencies which comes to everybody’s mind. For the moment we won’t need this feature. IOTA, beside enabling the exchange of value transactions (like other cryptocurrencies) has some interesting other features. As widely known IOTA enables no-fee based zero value immutable transactions. This service alone could be used to onboard all the data to IOTA’s network, but it is not organized (streamed) nor private. This is were a second level implementation Mask Authenticated Messaging (MAM) claims to introduces a secure and reliable streaming of data to this permissionless distributed ledger. The trick for enabling zero fee transactions lies with ending the separation of those who validate the ledger (called incentivized miners in other blockchain architectures) and those who perform transactions in the network. In IOTA you are both and if you like to add a transaction you need to validate two others before yours gets validated.
Sounds good. Lets try to implement the easiest solution for the M0-M3 milestones first and then explore the costs when scaling the use — case.
This article and the implementation of the use case mainly follows.
- https://www.iota.org/
- https://www.mobilefish.com/developer/iota/iota_quickguide_raspi_mam.html
- https://discordapp.com/invite/fNGZXvh
- https://github.com/iotaledger
- https://thetangle.business/
Certainly there is already an excellent open minded community which sets up many tutorials and Proof of Concept (PoC) cases available even though the IOTA Foundation is quite early to the party. I try to implement a solution based on the open source freely available protocol and point out where we would need to adapt when scaling up from our prototype.
Overview of implementation and results
The solution is based on
- Sending data with MAM. This is the MAM publisher program from our Raspberry Pi (RPi). MAM object creation and signing is done on device.
- Performing the tip selection (find randomly two transactions which needs validation called as a function getTransactionToConfirm (gTTA)) on a serviced Permanode for the cost calculation. Actually using the devnets full Node for the prototype in the implementation (which comes for free).
- Outsourcing Proof of Work (PoW) to the same full Node. (https://devnet.thetangle.org/nodes; https://powsrv.io/)
- Broadcasting it via the same full node which then gossips it to its neighbors to the entire network.
- Using MAM to fetch the data — MAM receiver — on a web application deployed on google app engine.
- Visualization via google charts (deployed on google app engine).
Most of the libraries of IOTA’s ecosystem are on github. Meaning we can use our package manager to quickly dive in.
As IOTA is permissionless we don’t need to create somewhere an account or get any other authorization to use the system. As it turns out the property “permissionless” is most exciting but as well most difficult to property to secure the network.
A much more advanced and complete prototype of a real MAM use case has been implemented and written down by Erwin Rooijakkers.
He listed already some of the current drawbacks like
- Non available out of the box secure authorization of your MAM channels.
Will most likely been introduced in the upcoming MAM+ release
- Non availability of permanodes (services) which stores the immutable data forever. (Drawbacks of having an immutable encrypted storage for ever and its implication of future tech potentially exploiting it.)
There is a service now available in beta (prices are subject to change) the estimation in this article only reflect the current pricing table — February 2019 https://thetangle.business
- Currently relatively high computational / energy costs for the Proof of Work (PoW) (used as a spam protection for the network) necessary before attaching a message to the tangle.
Can be significantly optimized using Field Programmable Gate Arrays (FPGA)
- Computational effort for the tip selection. The algorithm is necessary for finding the best two tips to be confirmed before issuing the transactions.
When it comes to the current relatively high PoW requirements we have some even more recent analysis available by Atis Elsts and by MicroEngineer.
The computational effort of PoW
Beside all the other tasks (Trytes to Byte conversion, signing, etc.) PoW holds the heaviest job for the device. Both show 83s to 90s for PoW on our RPi(not knowing which PoW implementation they used). Which would work for us as we just send every 3 minutes a new measurement.
The energy consumed for doing PoW
- Atis article derives approximately 55J (Joules) for the PoW which is 1.5*10^-5 kWh.
- Erwins article derives 0,00005kWh ~180J for one transaction for the PoW.
As far as I understood the computational time and with that the necessary energy consumed can be reduced to approximately 300ms at the moment using a FPGA instead of the RPi. Basically implementing the PoW routines with hardware gates to accelerate its computation.
Message size and TPS requirements
Assuming 1600 bytes per IOTA transaction (2673 trytes) and we need two transaction per MAM message. Further every device send every 3 minutes one MAM message (this is for our message 2 IOTA transactions).
Implementation #1 (currently possible)
- The Full Node is performing the PoW, gTTA and findTransactions (all heavy tasks outsourced — we need to trust the Node here).
- The Node just never performs snapshots (growth with the tangle network and all its transactions).
- MAM implementation without public key encryption (we just send the encryption key (side-key) e.g. by email).
This is my estimation based on thetangle.business service rates. Using getTransactionToApprove (gTTA) — the tip selection (0.3€ per 1000 executions) and then attachingToTanlge (PoW) (1€ per 100 executions). Further I have just assumed per user 100 fetches (findTransactions) from the MAM stream per day. We need for one MAM message (with the current payload and security) two transactions. Both transactions to be valid to be broadcast would need perform PoW. But we only need one gTTA for the MAM bundle .
This is a steep price and probably not worth to follow for our use case for now.
Implementation #2 (currently possible)
- Our RPi device is performing the PoW in ~90s / transactions. For our MAM message this would take ~180s . The energy costs would be with the device holder.
- The Node performs local snapshots after (let’s say 100 days) (data is lost after snapshots ) which means we lose older data. We just use for our cost estimation the permanode service from thetangle.business.
- MAM implementation with self-made key encrypted exchange (hopefully quantum proof).
Estimation is based on thetangle.business rates for gTTA and findTransactions. The PoW is done on the RPi and tries to approximate only the portion of energy necessary for PoW ( knowing that the RPi has to run anyway full time). For one device it’s assumed to stay in the always free limits but only cost the user in additional energy costs.
I stay positive that prices can be further reduced when using optimized HW and SW K1. As pointed out earlier it is all beta and early.
When it comes to the soft points I experienced some difficulties to figure out which libraries are up to date and can be used. Discord chat and the community really helps here when stuck K3. I particular liked the move from the foundation to restrict the chat to more development related topics rather than having it’s all about submerged in price talks on the IOTA token. Just recently the documentation got a new facelift too. The development talks are even live to be followed in the chat now - thats very interesting! The functionality which is provided out of the box is all available or getting build in variety of programming languages — but there is no real graphical user interface based platform yet .
When it comes to our particular use-case (visualization of environmental sensor data) its all a bit to much drama to store every measurement on the tangle and visualize it form there. But when you imagine it becomes necessary (by law) for hospitals to measure contamination levels — an immutable ledger for the audit trail might get more interesting (cost and value wise).
Implementation #3 (possible future setup — omega)
- The device is performing PoW in a very efficient way (maybe even only very limited amount, less than required today). Main protection will be Network-bound Proof of Work (NBPoW).
Bandwidth is a scarce recourse and naturally will limit the ability of bad actors as the costs of reserve a dominant part of the bandwidth becomes too expensive.
- Permanodes store all necessary transactions.
Either distributed (not all transactions on one node) with Iota Controlled agenT (ICT) - incentivized via IOTA tokens. Or centrally via IRI permanode — icentivised via IOTA tokens. At least that is my limited understanding of it.
- MAM+ used with out of the box sidekey safely encrypted exchange.
In order to fully enjoy this architecture I am afraid we will need to wait some years.
Implementing the use case
As outlined earlier IOTA is getting actively developed. By joining the Discord chat you can even see the Devs exchange information for there PRs on github. As well the second level protocol MAM is under development towards MAM+.
That means that the library I am using below might be in parts already outdated when you read this.
Transmitting a message on a public MAM channel
On the RPi
pi@raspberrypi:~/tmmiot $
mkdir IOTApi@raspberrypi:~/tmmiot/IOTA $
git clone https://github.com/jhab82/tmmiot-IOTA-agent.gitpi@raspberrypi:~/tmmiot/IOTA/tmmiot-IOTA-agent $
npm install
Before we are transmitting the sensor data lets shortly try to just send a message and understand what are the different options provided by MAM.
The above code initializes the MAM object that is necessary to create the MAM channel (For those who like to deep dive into this construct of Merkel trees go for it short or long)
We need the IOTA javascript library const IOTA = require(’iota.lib.js’)
which allows us to create transaction and outsource work to a IOTA Server (full Node). These Servers are distributed, mostly Virtual Private Server VPS (on AWS, Google Cloud or Azure) running IOTA Reference Software (IRI). The so called full nodes are the backbone of the current IOTA network.
IOTA was developed bearing in mind a mesh network environment of the future IoT. The current network tries to simulate the performance with measures like manual peering and computational PoW.
As we have not installed a full node ourselves we are using public available full nodes and its exposed API to attach our message to the tangle. And as we are in development state, we certainly use the IOTA foundation provided development tangle https://nodes.devnet.thetangle.org
Further we initialize our MAM object with the IOTA provider Mam.init(iota, undefined, 1)
and define it to be of lowest security and change it to be public. MAM can be
- public (meaning everyone with the address — we call it root address here — can read the stream) and
- restricted (only with the root-address and a password (side-key) chosen by the publisher) and
- private (only the seed holder).
That is already quite a bit of deep diving into the words and phrases used by the IOTA ecosystem. Don’t worry though we have not even touched the surface of this complex undertaking 😄 but the onboarding of data is very quick.
In the code we have an async publish function which translates our payload to trytes and attach it via Mam.attach
to the tangle. Here the full node takes the heavy job of finding the two tips to confirm (gTTA) performs PoW with those tips. On the RPi the transactions gets constructed (if private or restricted as well signed) and then send to be broadcast.
pi@raspberrypi:~/tmmiot/IOTA/tmmiot-IOTA-agent $
node mam_min_publish.js
It takes two IOTA transaction for our one MAM message. If you increase the size of the payload above the limits of an IOTA transaction (according to iota.org you can store 2187 trytes in this message) or change the security setting you would need more transactions per MAM message.
The tangle explorer shows us that we have created two transactions which got immediately confirmed.
In IOTA every transaction needs to validate two other transactions (those two transaction to confirm can be found by the tip selection or the getTransactionToApprove (gTTA) function). When it comes to zero value transaction they don’t need any validation though. Once they are broadcast to the tangle they get accepted. Contrary to value transactions which gets only confirmed when validated by others (basically making sure value is not created nor deleted - excluding double spends).
In order to protect the network from spam the IOTA network requires some Proof of Work (PoW) for every attached transaction. PoW needs some computational power and for now we outsource this to the public available full node (which luckily provides this via their API for free).
With the public MAM decoder we get the decoded message and the channel ID which is the root address and the next channel ID (next root address) where the next message would be stored.
The above screenshots shows us that we have stored our data successfully and that we can fetch it with the provided root-address (channel-ID).
What is different to the other implementation
Contrary to both other IoT platforms we did not authorize , register or had to set up an account nor give any credit card details for future billing — awesome you might think now, I’ll store all my data here. Unfortunately you can’t. At least not forever that easy. In order to keep the full nodes databases reasonable small — the database needs to get pruned from time to time (IOTA calls that a snapshot). Meaning only the balances of the IOTA addresses gets summed up and the database start all over again of course with sound consensus on the state of all non-zero balances. So the data is not immutable at all? Sure it is before the node gets pruned manually — or you have a Permanode which keeps all data and let you query it (most likely for a fee as in thetangle.business).
In addition to the other IoT platforms you wont have a central authority in this network which could view, alter or delete your data. Its is not Google or Siemens, I mean the administrator who authorizes your access. I am sure there are different approaches with sound rights management and encryption to limit the possibility of access in centrally organized permissioned systems, but they all prone to fail once we are moving to the machine economy.
Another features for the paranoid data owner in IOTA is its claimed quantum computer resistant encryption architecture. That sounds so futuristic and I honestly am by no means knowledgeable enough to discuss post-quantum cryptography with you. Anyways what I understood is that most public key algorithms are able to be broken by sufficiently powerful future quantum computers. One Time Pad OTP based architectures are not that easy to brute forced by very efficient quantum computing.
To sum it up we have
- created a MAM encoded message
- attached this message via remote PoW to the tangle
- received the message via a public tangle explorer (providing the channel-ID / root-address)
- decoded the message via the public web interface
When executing our above program again a new channel-ID will be generated which leaves our messages unrelated. But we like to be able to stop our stream and resume to the same channel when starting over again. Let’s do this next.
Resume publishing on a created channel
In order to resume publishing on a channel we need to store the next root address and some other information (including the seed) from the state of the MAM object (mamState).
In order to always restream the entire channel we need to store as well the first root-address. I have rewritten the code above to just do this. Now we can resume publishing on this public channel and would be able even after a our RPi breaks down to resume the channel.
Next we restrict the channel from others to be viewed — we need to change the MAM object to be encrypted by a chosen password (side-key).
Change the channel to restricted mode
There is only very little to be changed to the code except creating a key and transforming the key to trytes:
const key = iota.utils.toTrytes("tmmiot-iota-sideky");
Then we need to change the Mode of our mamState:
mamState = Mam.changeMode(mamState, "restricted", key);
That’s it from here we can’t use the tangle explorer but another explorer to view our restricted channel. Reviewing the mamChannelRoot.json which is created now with every new channel we have all the information to request the stream.
{"subscribed":[],"channel":{"side_key":"HDADADXCCDHDRAXCCDHDPCRAGDXCSCTCZCMD","mode":"restricted","next_root":"DKMVKBRGITNGLVEZPOGASILFTIRK9KFJHBKJESQKFIYMGOJLFQKWKFMGIPASCNNTBGR9ITJRQGYJSJILS","security":1,"start":1,"count":1,"next_count":1,"index":0},"seed":"UJHR9SDKXUOBNVHDGDXKPUBCABGPXDWAATCKITOTQRQSJJCBXONMZVDVKZDPPYXNZYLTQMHHJ9SISIDJF"}
When we resume the feed we get further entries.
Adding the sensor data to the agent
Just adding the package like we did for the other agents in Google and MindSphere.
const rpiDhtSensor = require(‘rpi-dht-sensor’);
//DHT sensor com package
var dht = new rpiDhtSensor.DHT22(4); //on GPIO 4const Sensor = require(‘sds011-client’);//PM sensor com package
const sensor = new Sensor(“/dev/ttyUSB0”);
// Use your system path of SDS011 sensor.sensor.setReportingMode(‘query’); //set the PM sensor to query mode
sensor.setWorkingPeriod(3); //set the interval to 3min.
And query our PM sensor. We get this every 3 min. published to our channel then.
Wonderful — we can run this nohup and leave the room and can rest assured knowing that we will have our sensor data secured, immutably stored in the tangle.
Just for remembering it — our RPi is not doing a lot of work though. Its fetching the sensor data every 3 minutes creating a MAM object from that and sends all the hardwork to the full node which provides us beside finding our transaction to confirm doing PoW and attaching the message to the tangle.
Probably the devnet full node are load balanced Virtual Private Server (VPS) — providing us with an entry point to the tangle devnet. But it could be as well a HW optimized cluster providing a very eco-friendly efficient service.
Visualization of data
Compared to Google Cloud implementation in Part 2 we have not yet seen an old school database which can be queried with SQL to get us all the necessary information for visualization various metrics. With MindSphere the onboarded data is not yet queried with available APIs or the GUI — we just saw a predefined visualization of our own sensor data. Both IoT platforms are not ready to just onboard new environmental sensors — they are fully permissioned. An administrator would need to authorize them (as shown). Of course we would need to automate the onboarding process but for now its all manually permissioned.
With IOTA everyone could already just onboard their data individually with above guidance to the same tangle. But in order to have all the data from all sensors (100, 1000 and 10.000) we would need to think a bit further.
Of course we could implement a market-place which enables to store the individual root-addresses and their side-keys (potentially by providing a bit of cash to the owner or better IOTA tokens) but that is maybe for the later chapters (similar to data.iota.org). For now we assume we know all the root-addresses and their side-keys.
Next I build a simple google chart application able to fetch the data from the tangle (by providing your root-key— similar to the firebase app we queried for our data but instead of listing the variables writing them to a chart.)
Visualization of the MAM stream
I have written a very rough visualization of a public MAM stream
Whereas in a database you can query the latest 10 entries of a filtered object (e.g. device). With MAM you need to run from the root until the end. As the next root of the message gets derived from the actual root.
I have streamed my sensor data one day to the tangle having now 480 payloads stored immutable in this distributed ledger. In order to visualize this and continuously update the stream I need to store somewhere my root-addresses otherwise I would need to run from the beginning to the end through the MAM which is computationally heavy.
There is already other MAM implementation in the making like RAAM. Furthermore as mentioned there is a project MAM+ on the agenda from the foundation which might come with additional functionality.
But here its the current MAM implementation. Or should I say I hope its the current. In order to reduce the computational effort we only derive the next message and store the the next root temporarily. The user can manually executes to decode the next message from that stream.
I have switched back to a public MAM stream (only because I don’t know how to securely ask for the sidekey on this backend implementation). I should have implemented it all client side (maybe next).
Code snippet from the backend function for fetching a single MAM message and the router.
https://iota-mam-vis-app-dot-tell-me-more-iot.appspot.com/