Dev Update February 2020

Weilei Yu
Weilei Yu
Mar 10 · 5 min read

Welcome to Streamr’s core dev team update for February 2020. This month, the team made a lot of progress on the Network and Data Unions (formerly Community Products). We recently published a blog post on a research study we conducted, with support from Wilsome, to learn more about user perceptions and expectations regarding the Data Unions (DU) concept. One of the example pilot projects built on top of our DU framework is, of course, Swash — check out their recently released roadmap for 2020. If you are interested in building on top of the DU framework too, or if you have some interesting use-case ideas, feel free to share them in our community forum and maybe receive a grant too ;).

Swash roadmap for 2020

Regarding progress on the Network front, we have recently removed a big roadblock in our long-planned, large-scale, network testing, comprising up to 2048 distinct nodes spread across the globe. The testing returned close to expected performance results, with 20GB of logged metrics to comb through for a more detailed analysis in the coming weeks. We are not yet 100% certain that the entirety of the test was flawless, so we are still analysing the logs to make sure there were no issues or misconfigurations. If that were the case, we may need to run another test with the required fine-tuning. The large network simulation testing was also the final missing piece needed for the long-anticipated Network technical paper. Hopefully, we will be able to share the results with you all in the coming quarter.

Another big update on the Network front is the implementation of WebRTC, to replace the current WebSocket model. Most of the functionality tests passed successfully, except for those related to storage. The next steps are: fixing remaining issues, then abstracting the implementation so it can be a drop-in replacement for the Network, without causing too much disruption. WebRTC offers some nice properties like NAT transversal, native P2P connection functionality, and is built for large scale data streaming.

WebSocket vs WebRTC comparison, source: Bloggeek

To make data transport more secure across P2P Broker nodes, we have been also doing cross-client tests for the next stage end-to-end encryption roll-out. The goal is to have a built-in key-exchange and a key rotation mechanism to enforce data streams permissioning, without relying on a centralized entity to coordinate the process. If you are curious to learn more about symmetric/asymmetric encryption and how it is applied to other sectors like VPN, for example, I found this article to be quite informative for non-technical readers too.

On the Data Unions side, we are making progress in multiple areas. Regarding the DU framework roll-out, our developers have been refactoring some of the DU server codes to make it more stable and scalable for the official launch. More specifically, we made sure that heavier calculations like Merkle tree computation would not block the server from receiving other requests. Additionally, we are trying to implement state-replication across multiple DU servers, so that we can handle a much higher concurrent connections load (hoping in the 10k-100k range). Our DU smart contract is currently being audited by external parties to make sure there won’t be any security bugs. Finally, regarding the progress of projects piloting DU, Swash is moving full steam ahead with their recently released roadmap for 2020, and MyDiem’s backend integration with the DU server is progressing well too.

MyDiem app built on top of Data Unions framework

What about token economics? It is not a secret that many community members have been very interested in the token utility incentive for the Streamr ecosystem. At the current stage, I believe most parties, both internal and external, have reached a common agreement that the strongest layer where we can apply a sustainable, long term token utility model is the Network, as an infrastructure service to all apps built on top of it, including Data Unions.

We are happy to announce that we have engaged the BlockScience team to assist us in the initial analysis of potential token economics models we could implement for Streamr. We will be hosting the first workshop at the end of this month. Of course, this is a complex topic that will impact our entire ecosystem, so we probably won’t come up with a solution right after the first session, but this marks the official start.

If you’re a dev interested in the Streamr stack or have some integration ideas, you can join our community-run dev forum here.

As always, thanks for reading.


Data Unions

Core app (Engine, Editor, Marketplace, Website)

News from Streamr

Data done differently. The open source, blockchain-powered Network and Marketplace for real-time data. #Ethereum #Web3 #SmartCities #IoT. Follow us on Twitter, join our sub-Reddit, or chat with us on Telegram.

Weilei Yu

Written by

Weilei Yu

Head of Developer Relations at Streamr. Vision driven and bias for action. Crypto enthusiast and learning about ML as hobby.

News from Streamr

Data done differently. The open source, blockchain-powered Network and Marketplace for real-time data. #Ethereum #Web3 #SmartCities #IoT. Follow us on Twitter, join our sub-Reddit, or chat with us on Telegram.

More From Medium

More from News from Streamr

More on Data from News from Streamr

More on Open Source from News from Streamr

More on Open Source from News from Streamr

Governance in decentralized networks

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade