Scaling CAP from Now to Next
Explore CAP’s architecture and our plans to break free from its current scaling limitations through stable storage & self-spawning buckets.
TL;DR:
We’re taking a look at CAP’s architecture and its limitless scaling ability via new upgrades coming up in the next months:
- By using horizontal scaling, CAP gets more scalable as more canisters get integrated.
- Using stable storage as the primary transaction storage point will put an end to upgrade failures
- Allowing buckets to spawn themselves will allow increasingly popular dApps to benefit from infinite transaction history being stored.
- Stay tuned for our upcoming IC-kit toolchain, a Rust-based & open-source canister development kit letting anyone write safe canister smart contracts.
Haven’t heard of CAP (Certified Asset Provenance) before? It is the transaction layer of the Internet Computer. Learn more about CAP here.
With the Internet Computer’s recent growth, we wanted to give an in-depth technical explanation of the architecture that allows CAP to scale and outline the future steps we’re taking in order to further increase its scalability.
Read on for a deep-dive. 🤿
CAP Scaling 101
Currently, CAP uses horizontal scaling, meaning that it becomes more scalable as more canisters get integrated.
The entire canister interaction takes place under the hood, through the CAP Router, without requiring extra input from users or projects. The Router is CAP’s orchestration canister and is responsible for entry point interactions with the CAP network. When your token contract integrates CAP, it communicates with the Router during the handshake process and initiates a CAP (Root) Bucket to start storing transactions.
Keep in mind that we do require the token or NFT contract to call the management canister and spawn a new canister, which will end up on the same subnet as the token contract. This guarantees instant and low-cost insert delivery to the smart contracts.
Storage Limitation
Currently, Cap Buckets can store up to 4GB of transaction history. This limitation comes from the heap storage provided to the canister. During upgrades, we write this data to the stable storage in a pre_upgrade, only to restore the data in a post_upgrade. However, the Internet Computer allocates 8GB of stable storage to canisters that we can take advantage of (hint, hint).
Despite the 4GB transaction history storage limit, the majority of dApp contracts integrated with CAP generally use only a fraction to store their transaction logs.
The only two canisters currently using more than 2GB worth of transaction history are the Sonic and WICP buckets. With almost 2GB left to spare, our team of trip advisors (aka the Psychedelic core team) has already run into issues — these two buckets when we were unable to perform the upgrade during our most recent attempt last week.
In its current state, the pre_upgrade hook on the IC should be able to finish the transaction execution in one block. However, we have discovered during our latest upgrade that one block was not sufficient to encode and write the entire transaction batch to the stable storage.
Scaling Solution
We believe that we’ve cracked the code to break free from the current limitations… 🔐
… and the answer is stable storage!
We will be moving to a stable storage first architecture in the upcoming months — we will use the stable storage as the primary transaction storage point and the heap memory allocation mechanism to only cache the query results.
Our team is also planning a feature allowing buckets to spawn themselves, instead of first calling the management canister. This way, if your dApp skyrockets in popularity and ends up running out of the available 8GB worth of stable storage, it will have its transaction history stored to infinity and beyond.
What’s Next?
We (PsychedelicDAO) are continuing to build our own toolchain for the IC called the IC-Kit. It is a comprehensive Rust-based, open-source canister development kit for the Internet Computer.
Once ready, the toolchain will enable anyone to write safe canister smart contracts and will help facilitate the management of persistent stable storage data structures.
For more information on how to start integrating IC-Kit in your canister development, check out our GitHub repo.
