Totem Tech Talks — August Updates

Totem Live is an open-source peer-to-peer accounting app built on Parity’s Substrate stack. Totem Live is now at a stage for early technical adopters to build, test and contribute our vision for the application. Totem Tech Talks covers development milestones on the path toward main-net. Tune in to catch up on our monthly progress!

August Totem Live Updates

📹 August edition (part one and part two)

Totem Live’s development has historically taken place on Substrate v1.0 to allow for a stable testing environment. This month, we started the migration process of Totem Live to Substrate v2.0. The accounting module was the first of Totem’s modules to be migrated to Substrate v2.0 and more will follow in the coming weeks.

Another significant migration took place last month. The application’s UI now uses PolkadotJS after previously running in parallel with Substrate’s original UI based on the so-called oo7 Bonds Library written by Dr Gavin Wood.

The switch was necessary for two technical reasons: PolkadotJS is required for Substrate v2.0 compatibility particularly around inclusion of custom types and because it brings with it RxJS. This gets us closer to a platform independent Progressive Web App also known as PWA for short which is a long-term goal for the project. Additional reasons for the switching to PolkadotJS are:

1. The oo7 Bonds Library is no longer being supported
2. Gains in performance from the PolkadotJS multi-query functionality for retrieving multiple data sets in a single blockchain read request

It also removed the dependence on a second connection to the blockchain which was an annoying side-affect of running the two libraries in parallel whilst we replaced the components employing subscriptions in oo7.

During this migration effort, we managed to completely brick our own chain (leading to some key learnings not mentioned in the documentation) around customizing and building your own UI supported by PolkadotJS notably around the hot-upgrade facility.

Here’s what we learnt and steps you might want to consider when building and testing your own hot-upgrade UI:

1. Make sure all the validators and authorities on your network are aware that this hot-upgrade going to happen. Get as many eyes on the project as possible before you pull the trigger.
2. Give yourself the opportunity to roll back. For example, turn off Grandpa in order to stop block finalisation. Substrate has a nice feature that allows you to roll back to the last finalised state — which would be before you switched off Grandpa. You can turn it back on again when you know that your chain is still functioning. If not, then roll back to the last finalised block effectively erasing any issues in non-finalized blocks.
3. Substrate has another good feature allowing you to export blocks to a certain block number. This is a pseudo-backup but should not be relied upon unless you can be sure that the validators, and boot-nodes will purge their chains and re-import all the blocks in the exported file. A re-import will make no difference if you have other nodes with a chain state that is further finalised than your hot-upgrade block. Note: If your network is significantly large this could prove to be difficult, if not impossible so this step should be considered for a worst case scenario.

You’ll want to avoid a bricked chain scenario — but this analysis might prove useful to some.

In our case, we used a badly processed WASM file to upgrade runtime from the UI. This is easily done because the raw WASM file is not enough for the runtime to understand critical information, like, how long it is. We suggest first reading the WASM hot-upgrade file and check its length. You might want to fingerprint the file at that point too — so that you can compare the file on the command line. Read it as an ArrayBuffer and call compactAddLength(). This does what you would expect and adds the length of the file to the file itself. This is so that the Blockchain knows how many bytes your upgrade contains and can extract the correct file. Do one last validation on the new length of the new file (it should have added 4 bytes) and this new file containing its own length metadata is what should be sent to the runtime.

This is a little UI message from Totem’s upgrade module to see what we do.

Screenshot showing how the Totem team validate their upgrades

One final point on this important information is that Substrate version 2 does a validation of the actual WASM file before allowing it to be included. If it isn’t a valid runtime the hot-upgrade transaction will fail, and your chain will not be bricked… but as we were not on this version yet, our chain had no such protections! Thanks to the team at Parity and in particular Jaco Greeff for his help solving this.

• If you are considering building your own Substrate UI using PolkadotJS, listen to the Tech Talks recording for more commentary from the team. 👉 Totem Tech Talks recording

Runtime Upgrade Announcement

Talking of upgrades, there have been many minor hot-upgrades to Totem Live’s blockchain runtime over the last month. The runtime is now at version 1.10.1. so if you want to keep you node running in native mode (and not WASM hot-upgrade mode, download the code or Ubuntu 18.04 binary from our repos and upgrade. As usual, join the Totem Discord channel if you have questions!

What were last month’s updates per milestones?

Follow Totem Live’s progress via our milestone tracker and tune in monthly for Totem Tech Talks. 👇

📹 New Tech Talks August edition part one and part two. Last month, we made progress on the following four milestones:

💎 Milestone: Live Meccano Canary Network

• Event processing and custom error events — which display meaningful errors communicated to the user and easier debugging for developers.
• Transaction ID — a new way to track transaction success or failure, even after a chain re-org
• Reporting currency added to user’s settings — now converts to any currency, and allows transactions in any currency.
• Cool, eye-friendly dark mode (Toufiq’s fav)

💎 Milestone: Live Lego Testnet (Substrate V2.0 migration)

• Accounting module — first to migrate, now completed.

💎 Milestone: Internal functionality

• New in-app chat interface allows for group chat, Totem Support chat and direct messaging. All communication can take place in Totem Live without ever having to leave the application. If you need questions answered just click “contact support” 👇 in the app and someone from the team will respond!

Screen shot from the Totem Live app showing chat for support buttons

• Migration to PolkadotJS for enhancements to front-end UI development and libraries

💎 Milestone: Accounting Engine

• Simple Pre-funded Purchase order mechanism (Blockchain and UI)
• Pre-funding Modules (Blockchain)
• Accounting Module (Blockchain posting)
• Pre-funded Task Invoicing (Blockchain and UI)
• Pre-funded Task Settlement (Blockchain and UI)
• Initial Accounting Statement overviews: Trial balance, profit and loss, and Memorandum Accounts Displays (Blockchain and UI)

Follow Totem Live’s progress via our 💎 milestone tracker and tune in monthly for Totem Tech Talks. Join the Totem community by:

1. Signing up for our mailing list
2. Joining Discord
3. Following us on Twitter
4. Testing out Totem Live application

👏 Thank you to founders Chris D’Costa and Toufiq Chowdhury for content input and review.