0Chain Weekly Debrief — December 1, 2021
Happy December! As we roll into the final month of the year, we hope you are all doing well. This week, we look into progress on app and website redesign as developers begin to implement the new UI. The blockchain team continues to conduct testing as they implement new fixes. Make sure to get your questions submitted by Wednesday December 8th to be included in the upcoming AMA with our CEO Saswata Basu.
Sculptex’s Use Case of The Week: Blobbers
“This week I’m going to talk about our storage provider nodes, known as blobbers. In particular, I look at how we will be able to cope with scaling the network quickly to cope with a surge in demand.
Blobber Requirements
A Blobber needs a good internet connection with sufficient CPU and memory to serve data that it might be required to store or retrieve. The recommended rig server specs for MSB miners/sharders had sufficient spare resources to also allow for Blobbing capacity. I already went into some detail about this in the community forum so you can read that if you haven’t already done so.
Active Set
So what capacities can we expect the network to have available on launch? Well for starters, there are over 100 Miners/Sharders. Each one has to provide at least 10TB storage to ‘seed’ the network, but most will be offering at least 20TB from the start. The standard rig spec we recommended has slots for up to 10 HDD slots although some compact units have room for 4. The rigs I am helping with have average usable capacities between 20–50TB. So multiplying this by 100+ miners we should have a minimum of 3PB on launch just from MSB active set, with scope to expand to at least double this just by adding more disks to existing rigs. (It’s a bit of a no-brainer to add additional HDDs to existing rigs since base costs are already covered).
Staking
9.5 million tokens have been pledged to enable the Active Set of miners and sharders for Fuji phase, but how much is gonna be required for the Blobbers? Although the testnets thus far have been focused on a 1 month allocation period, for mainnet, we are looking to launch with allocations up to 1 year. This will require a requisite number of tokens to be locked, equal to write price. Based on the recommended $10/TB/mo for 12months, each TB will require $120 of tokens for a 12 month allocation. Multiply this up by 5PB and this will require around $6million of tokens. Fortunately for anyone who is maxed out on their MSB stake, blobber storage can be delegated to (unlike the miners and sharders for the Fuji phase).
Note: that if someone offers a shorter duration, they will lose out on a lot of potential storage by those seeking a full 12 month allocation duration.
Big Blobbers
HDDs currently deployed in MSB rigs are mostly between 8TB to 16TB. Even with 16TB HDDs (calc 80% usable) and users selecting an EC ratio with 10 data parts, e.g. EC10/15, EC10/20, the max allocation size that could be achieved is in the early 100TBs range. To enable larger capacities and provide a smoother upgrade path to existing Blobbers, they will be able to span their storage over multiple disks incrementally adding disks to their rig. Unlike RAID disk arrays which are typically very difficult to adjust the size of once deployed, this allows Blobber expansion only limited by hardware constraints, although RAID arrays are also an option if a rig owner so chooses. Some might think that it is better to have multiple Blobbers per rig. This is still possible of course, but you will potentially lose out on storage from those seeking larger allocations, plus measures are being introduced regarding Blobber selection to ensure Blobbers are dispersed, so your rig will only really have 1 chance of being selected, especially if you are using the same IP address or subnet.
Scaling Storage
Although huge allocations are one use case (enterprise backups etc.) there will be those that have many files of different types. In these cases it might be preferred to have multiple, smaller allocations for speed of access, portability and Blobber diversity. In this case, smaller Blobbers with limited capacity still have plenty of opportunity to be selected. Additionally, although possible to keep adding disks (within hardware constraints), there will be a stage where other resources may become saturated so it would then make more sense to start a new rig.
Rewards
I have only really looked at Active set participants so far, but really, that’s just the tip of the iceberg. Updates to the reward protocol will also include a slice of block rewards to go to blobbers. So in addition to read/writes and interest, this will be an additional revenue stream. Exact details of this are yet to be released so expect a separate update on this soon! This is a great opportunity for those who missed out on a place in the Active Set to still establish themselves as Service Providers. In the future at Kilimanjaro phase and/or when the Active Set size needs to increase to cope with the BlockChain requirements, good-performing Blobbers will have a track record that can be observed that could encourage additional delegation.
Expansion
So how far can we scale? Well, there is no hard limit as such, but devs are exploring any potential bottlenecks that might arise when we start to scale, for example how we would cope with 100,000s of Blobbers! In this instance, the decision was made to split out the ‘random’ Blobber selection process from the miners to keep them as streamlined as possible. Of course there may be things that can’t yet be foreseen but this is why it’s so incredibly important that the team is already looking into how ‘upgrades will be implemented in the future without hard-forks’ as Saswata stated last week.”
Non-Dev Updates
The biggest developments this week are related to app development. Our UI/UX design team provided the finishing touches on the mobile storage app designs. Figma designs by the UI team are being sent to engineering for integration. They also submitted the first wireframes for web and desktop storage apps. Our website team has begun wireframing for the new block explorer and has submitted the general wireframing for the website.
Finalizing the mobile storage app is huge as this enables engineering to build out in preparation for mainnet launch. Additionally, building out the web/desktop in parallel will enable a full feature release of the storage app for mainnet without dealing with a “hurry up and wait” scenario as was potentially the case for engineering if they had been forced to wait for the UI/UX team to finish mobile before starting on web/desktop. The explorer development will be very important for investors and researchers who wish to analyze key data points such as token inflation, the network’s storage data growth, APYs for blobbing and staking, and storage costs relative to traditional cloud.
Development Team Updates
As noticed by some in the community, the cumulation of the past few weeks of blockchain enhancement and bug fix work merged to staging this week. Since then the team has been performing benchmarking against the optimized chain and we hope to share the outcome in future updates.
The significance of this milestone cannot be understated. Now that we have a stable baseline, we can target our efforts much more efficiently and perform any enhancements with greater speed. In addition, our system testing suite has grown to the point that it can be used to reliably detect most new issues being introduced to the codebase. The team has recently agreed on a POC to overhaul our pull request process, ensuring these tests are executed before any merge. Our confidence in our performance enhancements and internal testing is such that we may begin external testing with the active set in the weeks ahead — stay tuned for updates!
0Box and 0Wallet
While the blockchain layer continues to undergo testing, the 0Box and 0Wallet team continue to perform tests of their own. Following regression testing, developers pushed their latest changes and are ready for further testing and review. We have also begun collaboration with the UI team to implement the new designs, which have been developed over the past several weeks.
Blockchain
As the blockchain layer continues to undergo testing, the team continues to address known issues. An issue addressed this week includes miners not collecting enough VRF shares and thus not verifying the proposed block. This would in turn slow down notarization processes (as mentioned in previous weeks) and finally lead to the chain moving slowly. This issue has now been addressed. While continuing to make small modifications and continue testing, the team is researching methods to perform future blockchain upgrades including smart contracts.
Developer Resources
- Interested in learning more about building on 0Chain or becoming a service provider? Check out our GitHub for access to repositories. Community ambassador Sculptex has created numerous tutorials to help get you started.
- Try our BetaNet here! Users can create wallets and allocations, store files, send transactions, and share files.
- Need help navigating creating wallets, allocations, or joining as a blobber? Check out our documentation page.
- 0Chain’s API endpoints use simple and intuitive HTTP requests to interact with the blockchain in order to send/retrieve information to and from miners, sharders or blobbers in the active network.
About 0Chain
0Chain is a high-performance decentralized storage network designed to eliminate business threats such as censorship, privacy liability and data breaches. 0Chain helps entities achieve GDPR compliance, localization and tokenization, and monetizes private data sharing.
