Lisk SDK 2.3.0 Out, BFT Consensus in Development & Hub 1.20.0’s Ledger Nano X Support

Published in

Lisk Blog

11 min readSep 5, 2019

Dear Community,

The month of August saw our developer teams progress with the SDK and the Lisk Builders get busy with experimenting with our new toolkit. Here are the main takeaways:

SDK

Lisk SDK 2.3.0 released along with two patches (2.3.1 and 2.3.2).
The above releases are focused on the new partial mesh network topology.
We explore the Byzantine Fault Tolerance (BFT) consensus the team is working on for the next versions of Lisk SDK.

Lisk Builders

Lisk Center Utrecht community members Jonathan, Marc, Eric and Albert are working on Lisk.Bike, a proof-of-concept application built using the Alpha SDK.
Community members StellarDynamic, Tony, Koreben3, Lemii, Jackson Roberts and Matthew set up SideChain Solutions, providing tools, proof-of-concepts, and tutorials for the Lisk Ecosystem.
Moosty released their whitepaper for Lisk Directory 0.2, a sidechain project bringing greater transparency to delegate contributions.
Korben3 launched the incredibly handy Lisk Node Monitor Bot, a tool providing updates on nodes missing blocks straight to Telegram, making maintaining performant nodes more convenient.

UI

Lisk Hub 1.20.0 was released with the Ledger Nano X hardware wallet support.
Lisk Hub 1.21.0 has finished development, will contain German language support and a discreet mode.
Lisk Mobile 1.2.1 is out, includes a crucial BTC crash fix.
Lisk 1.3.0 is in development, includes various performance and UX improvements.

SDK

Last month, we released Lisk SDK 2.3.0 and two patches for this version (2.3.1 and 2.3.2) which fixed a few issues. Those releases are focused on introduce the new partial mesh network topology — we already described its features in the last Development Update, however we want to highlight a few improvements:

Requesting data only from outbound peers. Nodes will now only make requests to outbound peers. This reduces the attack vector, when many malicious peers connect to a target peer and occupy its inbound connections.
Network Ready Event. The P2P library fires an EVENT_NETWORK_READY event, which signals when the node has successfully connected to peers. The node can then start syncing and making requests to the network.
The default peerSelectForRequest function now simply shuffles peerList and selects one, instead of using a height histogram.
New class PeerBook handles peer management. We have new peers and tried peers lists, each of which has a specified number of buckets. Peers are added to these buckets based on a hash of their IP prefix and node’s secret. This makes a node more resilient against local attacks. This structure is influenced by a Bitcoin codebase implementation.
Peer eviction strategies. The buckets in each peer lists use various eviction strategies for newPeers and triedPeers that will defend us against receiving lists of invalid / fake IPs.

Partial mesh

The nodes are keeping connections with a subset of peers in the network instead of all of them. This will drastically lower the network traffic and number of connections and allow network to scale regarding number of nodes.

The graph below shows how the network will look like from the point of view of a new 2.1.x version node, with the older versions still in the network. The green ones are the new 2.1.x nodes whereas blue ones are older versions. The size of a node indicates the number of peer connections it has.

The second graph below shows how the network will look like with different versions of nodes. We can clearly see that the older versions tend to make a lot of connections and the new nodes are making fewer connections, which reduce both traffic and the number of connections in the network.

For the next versions of Lisk SDK, the team is working on implementing the Byzantine Fault Tolerance (BFT) consensus. Let’s explore the progress made.

Protocol Specs

We are making progress with the Protocol Specs tool which allows us to validate the protocol with the test data we generate. The tool is not part of the BFT by default, but it will allow us to validate the protocol changes that come with the new BFT implementation.

To give an example, the protocol tells us to invalidate a transaction if your account’s balance is insufficient. We can then use the Protocol Specs which is a set of generators that outputs JSON representing both positive and negative scenarios for our protocol. The JSON output contains the initial blockchain and account states, an input block, the final blockchain and account states (after applying said input). We can then feed this data to tests or other tools, in order to validate if any changes to the codebase break the protocol.

We are working on adding more scenarios, the full issue description with the list of tasks can be found on GitHub under issue #4087.

Replacing Rounds Module with DPoS

Currently, we are halfway through replacing the old rounds module with the new DPoS one. Some important functionality, like generating a list of delegates for a round, has been finalized and already implemented.

At the moment, we are replacing two core functions rounds.tick and rounds.backwardTick by the apply and undo variants exposed by the DPoS module. Again, full progress can be found on GitHub, issue #3686.

Processing Blocks

The new block processor is currently in the design stage and will be implemented as part of the BFT changes. The block processor is part of the chain module and should allow for more flexibility when dealing with different block versions.

We have come up with a pipeline pattern which allows to register multiple functions as part of each step in the block processing. Next, we can define a different block processor per each block version, in order to be able to validate different block versions. This means that the chain module is responsible for selecting the correct block processor for a particular block.

Block processor pull request, which is still under review, contains 237 commits and over 6000 lines of code changes.

Want to know more about our upcoming BFT Consensus? Read Nazar’s blog post exploring implementation of block finality. It’s also available on Hacker Noon.

Other Minor Changes

Filter has been added for account entity to allow searching in account.asset field: asset_contains and asset_exists — issue #3990
BlockSlots class has been moved into the DPoS module — issue #3789
Event is emitted once block is finalized from BFT module — issue #3908

We are looking forward to the next month where the new block processor will play a key role and allow us to work on replacing the rounds module completely, but also start work on the synchronizer mechanisms as described in the BFT LIP.

Reorganization / Refactoring of Code

New Utils folder is organized with files named after verbs (sanitize, validate, select, miscellaneous).
All the constants and events were gathered into a separate file at the root level to make them more accessible and make the code more readable.
Functions shared between newPeers and triedPeers lists were moved to a separate base class.
We improved the way the library exports its features through a root-level index file.

Lisk Builders

With the beginning of the Alpha SDK Phase at the end of July, the last couple of weeks were extremely busy for Lisk Builders. We had a whole range of projects being worked on, from Tic Tac Toe to Lisk Bike, as well as node management and development tools. Here is the run down for the month:

Lisk Bike

Lisk Center Utrecht community members Jonathan, Marc, Eric, and Albert are working on Lisk.Bike, a proof of concept application built using the Alpha SDK. Jonathan has been working hard and has already completed work on the blockchain and mobile application. With bicycles being a hot commodity in the Netherlands, it’s only fitting that the LCU community aims to build a JavaScript application and use the Lisk blockchain for the data registration. For the latest news and updates, follow the Lisk Center Utrecht on Twitter.

Tic Tac Toe

Community members StellarDynamic, Tony, Korben3, Lemii, Jackson Roberts and Matthew set up SideChain Solutions, providing tools, proof of concepts, and tutorials for the Lisk Ecosystem. First up to be launched was a Tic Tac Toe game by Korben3 along with an accompanying tutorial.

Lisk Directory

Moosty released their whitepaper for Lisk Directory 0.2, a sidechain project bringing greater transparency to delegate contributions.

EliteX’s Decred

EliteX have launched a new staking service for Decred as part of their exchange. They estimate that this could bring users a 12% ROI.

Tools

Korben3 launched the incredibly handy, Lisk Node Monitor Bot, a tool providing updates on nodes missing blocks straight to Telegram, making maintaining performant nodes more convenient. JesusTheHun also contributed Lisk Bootstrap, a tool using nvm and Docker Compose to smoothout setting up a local development blockchain.

PS: Always use third party tools with caution.

If you haven’t already, learn how to create custom transactions with Michiel’s comprehensive walkthrough. Meet Lisk Bills— a proof-of-concept invoicing application. You can also view it in webinar form.

UI

Lisk Hub 1.20.0 was released with the Ledger Nano X hardware wallet support

We have integrated Ledger Nano X hardware wallet support into Lisk Hub in order to enable users to sign in to their accounts using this hardware wallet, see the history of their transactions and make new ones. Ledger Nano X is a popular new hardware device and we hope users can benefit from this integration. The interface of this hardware wallet is pretty much similar to its predecessor Ledger Nano S. If you have used the previous model, you’ll have no problem finding your way around this one.

We have made significant improvements to how are our users notified about new releases. Ever since the first release of Lisk Hub, our users were automatically notified when there was a new release available with a pop-up window. Through the feedback from our users, we identified certain user experience flaws in the process. The main problem was that the pop-up window was distracting the user from whatever they were currently doing. To prevent this issue, we are now showing only a banner notification with a summary of the new release. This allows the user to finish whatever they were doing and only afterwards pay attention to the new release update.

Only once the user clicks “Read more”, they will be shown a modal dialog with the full changelog and an action button to install the latest release. This feature is currently supported on MacOS and Windows.

There are possible future improvements to the new release feature. When the user starts the download of the new version, we could show progress of the download also in the main window, not only on the app icon in the Dock. We could also work on implementing this feature on Linux because the support of auto updates of Electron-based apps on Linux is a lot better since we first implemented this feature. If you would like to see these or any other improvements, please let us know through Lisk Hub user feedback form, which is also accessible directly from Lisk Hub main menu.

Lisk Hub 1.21.0 has finished development, will contain German language support, and a discreet mode

We are also excited to present our very first localization of Lisk Hub tailored for the German market. From now on, all German speaking users can change the interface language to Deutsch easier usage. Hope you will like it, if German is not your preferred language, keep your eyes peeled — there will be more languages and currencies coming in the future. Similar to other configuration options, the language selector is accessible through the settings page.

We have also added a security feature to Lisk Hub — the discreet mode. If you’re a user of Lisk Mobile, you’re already familiar with this feature. Discreet mode enables you to view your account and manage transactions, while sensitive information like balance totals and transaction amounts are hidden. With this in mind, we add discreet mode feature and now all users can check their accounts without worry in public areas, all transactions amounts and balance are blurred out except those that are not related to the owner.

How Does it Work?

There are two ways to enable or disable the discreet mode: in Sign-in page and in Settings page. The discreet mode is disabled by default, meaning that if users sign in to Lisk Hub without activating this option, their account balance and transaction amounts are visible. But if they activate the discreet mode by checking the option in the sign-in page, all your sensitive information will be blurred out before you are navigated to dashboard.

Discreet Mode in the Sign-in Page

The option is located below the passphrase input fields, the interaction is through a checkbox and the text label is “Enable discreet mode when signed in (optional)”.

Discreet Mode in the Settings Page

The discreet mode option is located under the Security section.

Lisk Mobile

Lisk Mobile 1.2.1 is out, includes a crucial BTC crash fix

We have released Lisk Mobile 1.2.1. This version includes a patch that fixes a crash in the app which some users experienced when removing the BTC integration. With this release BTC integration can be added and removed without any unexpected behavior.

Lisk 1.3.0 is in development, includes various performance and UX improvements

For the past few weeks we have been working on the latest version of Lisk Mobile 1.3.0. This version will come with a number of bug fixes and enhancements that will improve the user experience. These changes include better behavior of the keyboard accessory button on iPhone X, a performance improvement thanks to an update on the navigation library, and a few other UX changes. We have also taken this time to improve the code quality of the project.