IOTA Permanode As a Service
Long-lasting data storage on IOTA’s Tangle
At first glance it may seem absurd that someone would need a service in order to have an IOTA node.
That’s because the activation of a known IOTA node is in fact a very simple operation, and in the coming releases the configuration of peering with other nodes will be fully automated.
However, node activation may be required for several reasons and some of these introduce additional complexities.
For example, if an application based on the IOTA network publishes data, being able to ensure node response times is a key factor. This involves an organizational cost and a specific expertise to keep the node efficient as performance needs increase.
Another typical requirement of such applications is to preserve this data over a long period of time.
The standard node actually deletes transactions from its database after a configurable period of time, and it does not provide any function to discriminate which transactions to delete and which ones to keep.
These two needs, which were directly perceived by us during the development of our other projects, motivated us to create the Things Lab Permanode service.
The Things Lab Permanode service
Things Lab’s IOTA Permanode service provides a fully managed IOTA node with “industry grade” features and the peculiarity of preserving the transactions produced by customers, according to their specific time requirement.
Each customer can create a specific application identity that will tag their transactions, allowing them to be precisely identified in the database.
The system has been developed on the Google Firebase platform and is therefore able to support virtually unlimited simultaneous connections, making it possible to guarantee the performance and reliability of the service.
In this first version, the service supports the following standard IRI API (IOTA Reference Implementation):
- GetTransactionToApprove: returns two valid transactions to link a new transaction;
- BroadcastTransaction: sends one or more transactions to other IOTA public network nodes;
- StoreTransaction: stores transactions on Google Firestore database.
Needless to say, the performance of the database is extremely high and is not affected by the number of connected customers.
Soon we will also release the AttachToTangle function, which runs ProfOfWork, thus providing all the API needed to create an IOTA transaction.
Data model
The call to StoreTransaction causes the transaction to be registered with its properties:
- Hash;
- Address;
- Tag;
- Bundle;
- Timestamp;
- Application ID;
- Private flag.
Note: Application ID and Private flag are additional parameters produced by the Permanode application platform.
Searching and retrieving transactions in the database
The following search functions are also provided:
- FindTransactions: given the parameters of one or more transactions (tag, bundle, address, etc.), it searches the database and returns all hashes of all transactions that possess the required characteristics;
- GetTrytes: gives the hashes of one or more transactions and returns the data structure in trytes.
The FindTransaction function has been extended to allow the search for transactions in a time range and by Application ID.
Please note that the FindTransaction feature of the Permanode service can only return transactions that have been recorded with the StoreTransaction feature. No transactions generated by other nodes and received via “gossip protocol” are stored in the Permanode.
In other words, this Permanode is a node that can guarantee the indefinite storage of your data, NOT all the transactions produced by the IOTA network.
However, all transactions stored by all Things Lab Permanode service’s customers are visible and can be retrieved by the FindTransactions or GetTrytes functions.
Virtual private node
For some applications, privacy is a key factor.
To support these applications, the Permanode service provides a special entry point to which it responds identically except for the StoreTransactions and BroadcastTransaction API.
By calling StoreTransactions on this endpoint, the transaction will be stored in the database with a privacy flag that excludes it from the GetTryes and FindTransacions responses from the owner of the transaction.
The BroadcastTransaction function on this endpoint does not perform any operations and avoids the propagation of the transaction to the other network nodes.
Obviously value transactions sent to this endpoint will never be confirmed.
This functional option is equivalent to activating a private network of IOTA nodes that exclusively share transactions created by an application. Given the virtually unlimited capacity of the system, this mode is more efficient and cost-effective.
Persistence of transactions
Normal IOTA nodes cannot store transactions on their local database indefinitely. This is due to various factors:
- Unlike other technologies, IOTA network nodes operate in asynchronous mode, so the incoming transaction volume is virtually unlimited and it is therefore impossible to guarantee indefinite retention of all transactions on all nodes;
- It is not possible to filter incoming transactions with some particular algorithm, because every new transaction could be fundamental to identify a conflict in the Tangle.
To limit the growth of the local database, the node then performs an operation called “local snapshot” which in summary removes the oldest transactions at zero value or with validity.
The deletion could be done selectively with application logic, but it would force to have one node for each application.
The Things Lab Permande solves this issue by combining the functionality of standard nodes with the virtually unlimited storage capacity of the Google Firebase platform.
The storage process is defined based on the needs of the individual client’s application, as each transaction sent by a client with the StoreTransaction function is stored in the Google Firestore database with its own characteristics (tag, bundle, address) plus two other parameters:
- Timestamp;
- Application ID.
A special function, RemoveTrasaction, allows each customer to delete a set of their own transactions that they no longer consider necessary by indicating a time period.
Customers can only delete their own transactions selected by the Application ID.
Architecture
The system consists of the following components:
- Registration front end: allows customers to activate a profile, purchase the service, check the amount of transactions stored in the system and perform searches;
- Application front-end: it consists of a series of Google Firebase Functions that respond to entry points by supporting standard IRI calls and extended service functions;
- IOTA Nodes: two or more IOTA nodes managed and monitored by Things Lab to which calls received from the application front-end are forwarded and must be managed by a standard node synchronized with the rest of the network. In particular, the GetTransactionsToApprove and BroadcastTransactions are addressed to IOTA nodes;
- Performance monitor: it is a service active on an external vps that simulates a user and constantly checks the operating status of the system, alerting the Things Lab team in case the response times are higher than set limits or IOTA nodes are not synchronized.
Service and storage costs
The service will likely be priced with “Service Tokens” that can be purchased in packages of different sizes.
These Tokens can be spent to purchase the storage of transactions and using API functions.
Note: these “Service Tokens” will only be an internal currency for the service, they are NOT a form of cryptocurrency and they will NOT be traded or sold to the public.
More details about the pricing and economic conditions will come soon.
