A Conversation With Streamr
A novel way for people & machines to trade data on a decentralised network
Real-time data is increasingly turning into a commodity. Huge volumes of data are being generated in line with the growth of IoT and the ubiquity of smart devices. In the global IoT market, IHS Markit forecasts that the installed base will grow from 15.4 billion devices in 2015 to 30.7 billion devices in 2020 and to 75.4 billion in 2025. Much of this data is fundamentally valuable.
Alongside this development is a megatrend in motion towards the next generation of the computing stack — Dapps. However, these Dapps do not run in isolation. They need external data to function. As it currently stands, storage and distribution of real-world data remain centralised, and Dapps therefore remain liable to all the known problems associated with centralisation: Concentration of power, lack of robustness and a vulnerability to cyber attacks.
What is needed is a natively decentralised data backbone as a complement to decentralised apps.
This is where Streamr comes in.
Streamr delivers unstoppable data to unstoppable applications. It is the real-time data backbone of the global supercomputer. A decentralised network for scalable, low-latency, untamperable data delivery and persistence, operated by the DATA token.
Anyone — or anything — can publish new data to data streams, and others can subscribe to these streams to power Dapps, smart contracts, micro-services, and intelligent data pipelines.
Ethfinex spoke with Streamr CEO, Henri Pihkala, to learn more about the Streamr team and their plans for a decentralised data marketplace.
Q: What is Streamr?
Streamr is building the world’s leading marketplace for real-time data. On the technological end, a successful data marketplace requires supporting infrastructure and analytics tools, both of which we provide.
We believe that creating a decentralised crypto powered platform to trade the world’s information will benefit everyone. Companies will be able to better build and market their products in the Internet of Things (IoT), people will finally gain control of the data they produce and society as a whole will be better off when the value in the data is not concentrated in the hands of a few giant companies. Those are our ultimate goals.
Q: How and when did the idea for Streamr originate — What prompted it? How did you guys get together?
Back in 2011, a small group of us created an automated trading platform. We put it in the colocation facility of a major stock exchange and for a year, the platform worked its magic. Although our efforts paid off, the experience left us unsatisfied. The world was full of real-time data sources, which our technology could help visualise and make valuable, and yet we were stuck in a stuffy stock exchange basement helping a small number of people. That’s when we decided to create Streamr. Those first iterations were centralised and aimed at the cloud SaaS market; however we soon realised the superiority of a decentralised version and voila, Streamr Network AG was founded.
Q: How has your vision for Streamr evolved over the past few years, as decentralisation, Ethereum and IoT has gained traction?
Decentralisation is the key! Back when we were working on the centralised cloud version, we were able to land companies from small startups to big enterprises as clients, and build solutions for their real-time data-driven problems on top of our platform.
While this was all good, we were passionate about the idea of a data marketplace. We felt that the centralised world, with all its middlemen, complex payments and trust-requiring ecosystems were not the way to go. In 2016 we got excited about the new wave of decentralised technology — not just blockchains and smart contract platforms — but also other decentralised protocols and services that could be powered by a token ecosystem. We realised that a decentralised marketplace would solve most of the problems of a centralised one, and have better network effects. The marketplace would bring data buyers and sellers together, a scalable messaging layer would power the real-time apps of the future and an accompanying blockchain would handle identity and payments, thereby connecting Streamr to existing decentralised technologies.
Q: Can you provide a low level description of the architecture behind Streamr and the interaction between the visual editor, engine, network, marketplace and DATA token?
Sure! I’ll start with what the components are and how they interact. The overall Streamr system consists of infrastructure and applications. The infrastructure includes the Streamr Network and a companion smart contract platform, currently Ethereum. The Engine/Editor and the Marketplace are two applications connected to the infrastructure.
The Streamr Network is responsible for data delivery and storage: When a data producer publishes a data point it gets stored as well as delivered to valid subscribers also connected to the network. It’s like an instant messenger for machines, with both free and paid chats available.
The Streamr Marketplace is a view into what’s publicly available on the Network, listing both free and paid content. Anyone can connect their data source to the Network and start offering it as a product on the Marketplace. Similarly, anyone can go onto the Marketplace to search for data streams they need, and purchase access to them.
The Engine is an application for applying computations to real-time data streams. It can ingest data from the Network via its APIs and turn the data into visualisations, automation, interactions with smart contracts, or even enriched data that can be produced back to the network and potentially sold! The Editor is a visual programming environment in which computation processes for the Engine can easily be created.
The DATA token has two main functions:
- It is a means of payment on the Streamr Marketplace
- It is needed for the incentive and reputation mechanisms throughout the decentralised Streamr Network.
The incentive system rewards people for running nodes that participate in the Streamr Network — kind of like mining in blockchains, but instead of solving an artificial CPU/GPU hashing problem and wasting energy, the nodes contribute useful bandwidth and storage to collectively produce the messaging services of the network.
Digging deeper into the tech, it is important to understand that there are milestone versions of Streamr with increasing levels of decentralisation. This approach allows us to provide a production-ready system from day one, while iteratively working towards full decentralisation.
The roadmap for the Streamr Network component serves as a good illustration: The current Streamr Network is still largely based on the cloud infrastructure, utilising open source data frameworks such as Apache Kafka and Cassandra. The next major iteration will be a peer-to-peer version of the network, utilising the libp2p networking stack, with the nodes still run by us. After that, we’ll proceed to a version in which anyone can run a node.
Q: What are the go-to use cases for Streamr, and what type of applications do you see benefit from Streamr?
I’m going to give you a broad overview of three categories of use cases we’ve identified.
- The first category is a fairly simple. Once our Marketplace comes online this year, we expect many companies who already sell their data in closed, private, fiat-driven data markets to head our way. We’re another shop window, if you will, for many of those data retailers which currently includes the financial sector, weather, marine services etc. This isn’t going to change the world as such, but it will make things more convenient for the parties involved. The bit that will be exciting is that when all this data is in one place, which types of apps will creative data specialists and developers be able to create?
- The second general use case category is even more interesting — it is what happens when we liberate all that realtime information which is currently siloed, isolated, being junked or not even collected, because those companies haven’t figured out ways of monetising it or making it useful. It has either been too inconvenient or too expensive to set up bespoke systems to do so, or there has been no obvious way to put buyers and sellers in touch with each — other until now. On that note, we’ve got people in South East Asia talking with us about how to manage fishers (governments and environmental agencies and charities subscribe to sonar data from fishing fleets). We’ve also got medical equipment companies chatting with us about creating new models to lease equipment because they can now stream the data P2P through a neutral and open platform.
- The third category of use case is the IoT sphere. The IoT economy is still a long way off from being fully fledged. But we’re preparing for that and we feel that customers are going to want to own not just the physical elements of their gadgets and appliances, but also the data they produce. People might have gotten used to the Facebook paradigm; they generate data to use the platform freely and that’s what keeps Facebooks shareholders in business; however, are they really going to be happy forking out for a luxury car, or a smart trainer, in order to create value for someone else? Forward thinking companies are already approaching us about these issues. They want to liberate their customers and think Streamr’s Marketplace will help them do it.
Q: What is your relationship with other projects inside and outside the Ethereum ecosystem? Which are you the most excited about, both for Streamr and otherwise?
One of our big collaborations is with Golem. The team in Warsaw have incredible technical insight into building a decentralised service layer based on a P2P network, which is what we are doing with the Streamr Network.
Clearly what is also important for decentralisation as a whole is getting a stack of technologies that can integrate into each other seamlessly, removing centralised weak points. Whilst no one project is irreplaceable, applications like Golem’s decentralised computing, financial systems (OmiseGo), or improved browser integration (MetaMask) need to happen for true decentralisation as a whole to be both realisable and a pleasant user experience.
At the end of the day however, for middleware projects such as Ethereum and all the other ones mentioned above, it’s all about the use cases. To name a few examples, we are collaborating with Zipper, a project building a blockchain phone, which can act as a data source for Streamr to help monetise personal data; and REGEN, who are applying IoT in agriculture to produce and tokenise data on carbon dioxide sequestration. There are other interesting partnerships in the making, both on the R&D front as well as with data producers seeking to participate on the marketplace.
Q: Is there anything you want to communicate to the Streamr community?
Yes! There have been a lot of promises made in the space and if there’s one thing we like to stress, it’s our record. We already have functioning, working applications — our Editor and Engine — and so we’d love to invite even more developers to play and build with the tools we’ve created.
Our user numbers have grown more than 700% to 8,600 over the last four months but I don’t think there’s any better way to assess the credibility of a tech team than by looking at what they’ve built. As they say — the proof is in the pudding.
Q: What’s next for the Streamr team?
The big thing for 2018 is that we want our Marketplace to be up and running, with people buying and selling data on it, by the end of the year. This will also mean real utility for our token, DATA. Whilst a number of other projects are focusing on tomorrows world of IoT, we want to make sure our Marketplace services todays Information Economy, which is worth billions, making a few companies overly powerful whilst the rest of us get nothing for the data we generate.
Once our Marketplace has been road tested with traders in todays data economy, it will help us expand to encompass information traded between machines and connected devices themselves.
This interview is the second in a series intended to share and foster transparency throughout the Ethereum ecosystem. You can read our earlier conversation with the Cofound.it team here.
Do you have an idea for a project you’d like to learn more about? Let us know.
