Market Makers: A retrospective on Streamr’s real-time data Marketplace
We launched the Streamr decentralized Marketplace for real-time data back in late May and since then, we’ve had lots of insightful questions from our community about how it’s been going. Since June I’ve personally been in charge of developing the Marketplace it from a business perspective so this blog is an attempt at being as open as possible about what we’ve learnt from the past four months or so and what might be done in the future to improve the Marketplace in terms usability, and increasing the number of people adopting it as a platform for buying and selling their real-time data.
I’m an avid proponent of continuous improvement and learning and so the past four months have taught us a lot. A rough assessment is this: we’ve made progress in some areas but in other areas we’ve stalled or hit roadblocks. But all of this is for the good because it has taught us a lot about market dynamics related to real-time data as well as more in-depth insights about the Marketplace’s functionality.
Let’s start digging through these insights in more detail:
Companies want to monetise data
One of the most positive things we’ve seen is that companies that are already monetising data on one platform are pretty willing to monetise it on another. Data providers aren’t fixated with just one channel for data monetisation. As the effort to integrate data to a new marketplace decreases, it’s rational to maximize the number of marketplaces a data source is listed on. The reasons why data providers want to join the Streamr Marketplace are divided between the exposure we can offer, the new revenue channel it enables as well as the ability to try out blockchain technology.
Pricing
Pretty much every data provider we attempted to onboard to the Marketplace had one recurring question: “What’s the right price for my data?” Even for companies that already monetise their data, the question was still relevant, since the Streamr Marketplace offers the benefit of microtransactions. Whilst most vendors have been used to selling their data for a minimum of one month, Streamr now offers providers the opportunity to vend their data for just one hour. (In a future of Machine to Machine transactions, we expect this could even be reduced to seconds or based on events received).
There is never going to be a definitive answer to that question. As obvious as it sounds, things are worth, what the market is prepared to pay for them. We will probably be able to give a better ballpark figure in the future as we maintain discussions with an ever increasing number of industries, companies and individuals.
We have also gained some ideas to help the discourse between data providers and potential data buyers. One of them is the community product and another one is data bounties.
Data bounties will allow potential buyers to list their their willingness to purchase certain type of data with a certain price point. Data providers could also set their data tentatively on display to spark interest from potential buyers. With the Community Product, data buyers will be able to group data from similar products (say, all Fitbits in France) and buy the data from those vendors in one go. It will be an exciting development as the payments can then go to back to the individuals directly and this will help fulfil our overall mission of putting people back in control of their data. I say a little more about this below but here is our current diagram of how we expect this pattern to function:
Integrations
As the onboarding process continued, the next step was to integrate producers’ data to Streamr. The key learning in this phase has been that while data producers are generally willing to start monetising their data and add additional revenue streams through the Marketplace, many are hesitant to implement the integration themselves or unwilling to keep the integration running on their hardware due to incurred costs.
Initially, we thought that the problem was a lack of quality information/supporting tutorials for building integrations. After we created a fairly well-prepared set of instructions for both the data integration as well as product creation on the Marketplace, we were able to remove that concern.
To further battle the problems of integration, we have now started to implement integrations to help data providers as well as keeping those running on our infrastructure. We have also implemented a native integration to the IFTTT platform for the simplest use cases, such as polling an API with certain time intervals.
In the future, we are planning to release example integrations and ready-made templates to ensure that everything goes as smoothly as possible during the integration process. It seems there is demand to try things out as fast as possible with more or less copy-pasting an example and retrofitting the data. We also hope that our Editor will also help service this purpose. We are currently working on refactoring it and we hope that a second iteration in the near future will make the integration process even more convenient.
As we started to build the integrations, something we hadn’t anticipated happened. Implementing the integration for data providers means, all of a sudden we get direct access to their data, and thus we become a data redistributor. Typically data providers have a click-through agreement or in some cases even a physical contract. It’s typical that these contracts are very strict and prohibitive by nature. One of the restrictions usually present in such contracts is prohibiting redistribution of the data, which is exactly what we are doing if we implement the integration.
For the meantime we have overcome the problem using a memorandum of understanding in order to protect both parties from the misuse of the data and anything else that could jeopardize the relationship. In the future, as the Streamr platform becomes more decentralized, Streamr will no longer need to be a party in data redistribution agreements.
Real-time data Editor
Sometimes the data pushed to Streamr is not in its final form, but refinement or enrichment is needed before the final data product can be pushed out to the Marketplace. Streamr has a drag & drop Editor for integrating and manipulating data. While the tools in the Editor provide a nice way to build simple visualizations and logic, we’ve learned that it’s sometimes faster to use plain old programming to get things done. If you want to clean the data or do conditional logic, may be faster to write a couple of lines in Python, JavaScript, or any other language you’re already familiar with than trying to utilise the various components the Editor provides.
The projects built using the Editor are called Canvases, and they run on our event processing Engine. In addition to the work-in-progress usability of the Editor, there was a bigger underlying problem in the Engine: stability.
Due to the instability of the platform in its current state (this has been mostly solved now), the Canvases occasionally stopped running. When stops occurred, product owners would have to manually intervene and often periodically check whether their product was still up and running. This has caused at least couple of gaps of information in the data products, which is a shame. Going forward, unhealthy Canvases will be automatically detected and repaired.
The Editor and the Engine are still centralized services running on standard cloud infrastructure. While we have taken steps to improve them and overcome the aforementioned problems, the main goal is to eventually create fully decentralized services out of them. When this happens, some of the problems would be solved by decentralizing the computation. The short-term plan however has been to fix the glitches by making the system more stable and production-grade while adding small usability and styling improvements.
Product creation
After the data integration is done, the data is flowing in, and in its correct form, things should be simple right? You just create a product and that’s it. Well, not exactly. There are a few more hurdles to overcome. While the Marketplace user interface itself is wonderfully straightforward to use, product owners (and subscribers) have to deal with new technology, namely cryptocurrencies and smart contracts. This means creating a Marketplace product is not as user friendly as it could ideally be.
When you are creating a paid product in the Streamr Marketplace, you need both a cryptocurrency wallet and some ETH. The Marketplace utilises smart contracts for both storing the different products in the Marketplace as well as the individual transactions between data buyers and data providers. Smart contracts consume ETH as gas, i.e. you pay for the computation that will be executed within the nodes. In order to pay the gas needed for the product creation, data providers need ETH.
For many, using and interacting with cryptocurrencies is unfortunately still a pain in the rear. It requires above average technical knowledge on how things work under the hood. Even installing a wallet such as MetaMask is not straightforward (though their recent product update is a real improvement in UX terms).
To overcome this barrier of entry for cryptocurrencies and smart contracts, we have come to the aid in some cases by supplying the necessary ETH for the product creation. While services, such as https://ccswap.myetherwallet.com/ exist, we have sent ETH to the data providers manually, just to make it as effortless as possible. Obviously this isn’t feasible over the long term but it helps us “getting things done”.
Realistically speaking, some of the problems that relate to crypto usage are out of our hands. Wallets need to become more user friendly by hiding the technical aspects and focusing on the essentials. Calculating the right amount of ETH to use as gas should also be addressed either by wallet creators or with Ethereum protocol level improvements. But these are space-wide issues which Streamr can’t solve on our own.
Where are the buyers?
Since its launch, the Marketplace has had on average one transaction per day. A transaction means either a data product created, a subscription purchased or a product deactivated. This clearly isn’t good enough. And although we have had somewhat positive reception from data providers, we haven’t yet paid too much attention to the data buyers. This is something we are going to change.
We are currently targeting potential data buyers and are now offering a bespoke matchmaking service to find suitable data they can buy and utilise to achieve competitive advantage. We’ll have good news about this shortly but this effort consists of both learning ourselves as well as educating others in what can be achieved using real-time data. This brings us to the next important insight, which is that the most valuable real-time data is not yet being monetised.
Companies that are not yet selling their data
Before we launched the Marketplace we set up a sprint operation to contact data providers who were already monetizing their data. Most of the current data providers are from the financial industry. While there are interesting opportunities in this particular vertical, there’s an unfathomable amount of real-time data sources in other verticals. If monetised, that could disrupt many traditional ways of doing business. It seems that in many industries, few people know about the opportunities to monetise data. We have started to tap into IoT, and the automotive industries more strongly to educate organisations within those verticals and better explain the prospects that monetisation of real-time data can enable.
Individuals
Streamr’s core mission is to enable individuals as well as companies to take control of their data. While there are currently some paid data products by individuals such as local weather readings from Blumenthal in Bremen, Germany, there hasn’t been yet a rush of individual data contributors. Because of this we are building a feature called Community Products that will essentially enable easier pooling of data together from multiple sources. We have ongoing projects together with our partners where we are generating data from multiple sources, whether they are air quality sensors or mobile phones and we’ll release more information about this soon. Pooling it all together and allowing buyers to purchase those individual data streams as one conglomerated product through the Marketplace, and then distributing the profits evenly between the data contributors will really set the Marketplace apart from other data marketplace offers out there and helps realise one of the big promises of crypto — scalable micropayments. The Community Product once further developed will also allow smart gadget manufacturers to give their customers the opportunity to earn $DATA in return for permissioning the sale of their data on the Marketplace. We envision these kind of initiatives are going to be a massive driver for Marketplace adoption.
Summary
Since the launch of the marketplace we have learned that there is a supply of many kinds of data. It’s still early days and things take time, but the team is undoubtedly moving forward. We have shifted our focus to the demand side as it’s now essential to ensure that there are buyers for the data, even before we find the corresponding data on the seller’s side. We will have a little more news on that front in a few weeks. Innovative initiatives such as the Community Product will propel the Marketplace to its deserved upward trajectory and as long as the crypto space at large continues to make payments easier, we should eventually reach our goal of making the Marketplace a stand out platform for the new real-time data economy.
