Natural Sharding within the OpenPoll Blockchain

OpenPoll is a blockchain based platform which innovates the polling and consumer research industry- you can read more about what we do at openpoll.io.

Blockchain technology has evolved rapidly over the past few years and many new and innovative solutions have been proposed to solve large problems preventing mainstream adoption. Inarguably, one of the largest problems blockchain technology faces is scalability. With every node being privy to every transaction, there is a natural limit to transaction processing. Lots of temporary solutions such as increasing the block size seem obvious, but are not sustainable for long term network demands. This article explores the use of natural sharding within the OpenPoll network to address scalability concerns.

OpenPoll requires a large data throughput and high amount of transactions-per-second (TPS) in the form of poll and survey responses. Unlike traditional cryptocurrencies using blockchain technology, OpenPoll has a few unique characteristics:

• Responses can be significantly large (when users provide text and multimedia responses), and quite frequent.
• In a cryptocurrency transaction, you must analyze blocks 1…n-1 to process block n (this is optimized by Merkle trees caching block states) to sufficiently ensure a user does not double spend. With a given response, you do only need to validate (1) the user has not previously responded, and (2) the response is originating from a valid user.
• Transactions are not intended to be used as a currency, and our stakeholders do not care about speedy payment, as such, transaction times are relatively insignificant.
• Each poll and survey is discrete from each other.

We have designed our blockchain implementation to specifically exploit the features specific to our problem in order to develop the best-possible implementation. Using existing blockchain technology, which does not cater to the specific problem OpenPoll is solving, would be inefficient and ultimately unable to adequately perform.

Poll & Survey Sharding

The OpenPoll platform is designed to have a primary blockchain which is the ‘master record’ and contains references to polls and surveys, as well as serves as the ledger for the POL token. Each poll and survey runs on its own discrete shard which itself is a blockchain. Through a proper incentive structure, and exploiting miner greed and personal interest, we ensure all poll and survey shards are sufficiently mined. The natural method of sharding utilized on the OpenPoll network is only possible based off the unique characteristics of OpenPoll. A few benefits of using this method to shard include:

• Miners can directly participate in shards with low computational requirements without participating in mining pools, resulting in enhanced redundancy and higher decentralization.
• Ability to validate and confirm relevant polls and surveys without requiring (i.e. downloading) all data stored on the OpenPoll network.
• Scalability limitations restricted to single polls and surveys and not across the entire platform.

Shard Transaction Fees

Polls and surveys on the OpenPoll network are deterministic in size: a set number of responses, a fixed data retention time, known response payload acceptable byte range, and a predetermined response timeframe. This allows for platform fees to be computed at the time of poll and survey creation, and assessed directly to the creator to fund the shard for its entire lifetime, up front. Funds are transferred from the creator into a locked account where they are distributed to (1) shard operators, and (2) respondents, via smart contract (embedded logic). This ensures that poll and survey creation is irrevocable, and wallet fund availability computations remain light and sustainable- no gas required (i.e. deterministic through Merkle tree opposed to computing outstanding polls at each transaction).

Mining costs are assessed as a fixed percentage of the total poll / survey cost, rather than user designated or optional. Considering the OpenPoll network is significantly more cheaper than traditional market alternatives, assessing a fixed percentage network operation cost to fund miners will not deter platform users. This results in a large amount of funds paid directly to network operators, making the OpenPoll network extremely attractive to honest miners- especially considering small computational devices can remain competitive.

Shard Retention

Data stored on shards is not permanent and will only exist as long as the shard exists. This is a fundamental requirement for sustainability as the relevance for polls and surveys naturally attrit. Storing irrelevant data is unnecessary and expensive. To ensure we account for relevance attrition within our platform, we allow for relevance to be asserted within the blockchain in two ways: (1) an interested party can act as a shard miner and ensure shard existence, or (2) additional shard funding can be added to extend the shard lifetime. Poll and survey creators provide adequate and sufficient funding for shard duration during creation, and can optionally extend lifetime if they deem necessary.

It is necessary that shards be verifiable even when no longer operating. At termination, any user deeming the results of a poll / survey relevant can preserve the shard’s blockchain. A shard blockchain can be validated as a hash of the shard chain is permanently preserved in the primary OpenPoll blockchain- it should be duly noted that a hash of the completed shard chain is different than the data contained within.

I hope this article has provided more insight than that available in our white paper for how we are addressing scalability concerns, particularly regarding the large amount of data involved in poll / survey responses. By leveraging the unique characteristics of the problem we are solving, and building our own blockchain, we are able to solve pressing and unsolved blockchain problems not solved for more generalized applications.