Oyster Development Update June 22, 2018

Today’s development update will focus on the work completed during the week of June 17–22.

The development team has made further progress in optimizing the web node script. The previous web node build had a total size of 1.8 megabytes, which is a significant amount of JavaScript to parse through (this was an issue originally submitted by an Oyster community member). The development team has since taken this concern from the community into consideration and has managed to reduce that build to 575kb (a decrease of 70%). By reducing the size of the web node script, the response time for clients will be much shorter, as the necessary data to be downloaded client-side to run the protocol is now significantly smaller.

In addition, the development team has made significant strides in the Ethereuem-related functionalities of the broker nodes, as well as fixed bugs related to gas limits and nonces. Furthermore, QA has been completed on the bury_treasure_addresses task. This task does three things (and for the interest of our community, this post will include the Etherscan transactions that show this task in action).

First, the treasure is sent to a treasure address (https://etherscan.io/tx/0x896b8d39ae6b1e89f8b931e5a6a49a12e33922c3d62290fc94b9b2c0582579db).

Then, a tiny amount of ETH is forwarded to the treasure address to be used as the gas for the bury() call (https://etherscan.io/tx/0x54233b4b520bc7634475b786a2306727a5c7dbe737c939508db9550b56c1a2a9).

Finally, the eth_gateway method is invoked and calls the bury() method on the Oyster PRL smart contract (https://etherscan.io/tx/0x0a7f9427d61e8a4040d5b428f0042996f1ea16a58c7baa23a0174b4b7be6c174).

The transaction ids provided detailing the bury_treasure_addresses task are examples created during testing; thus, the gas prices used in these transactions are relatively high to ensure the transactions are confirmed within a reasonable time frame. In production, these gas prices will be much lower. With the bury_treasure_addresses task working correctly in testing, the development team has moved on to testing the claim_unused_prls task, which will invoke some of Oyster’s eth_gateway methods so that the broker can retrieve any PRLs that were leftover and not needed for treasures.

The development team has also been working on finishing the design for ‘rev2’ and writing proof-of-concept code in Python to use as a reference implementation in other languages later on. The team has started on documenting how rev1 (the current build in production) works to support legacy clients in the future as well. ‘Rev2’ is an exciting future improvement to the Oyster Protocol, as it will give the protocol the capability to offer multi-file uploads in a single session to a single handle. Moreover, each file has the potential of having its own password that the user needs to remember on their own (completely optional feature) and also enables features like media streaming due to a change in how treasures are buried. The design changes in ‘rev2’ of the Oyster protocol will ultimately help wide-spread adoption of the protocol for commercial-grade data storage, as well as lead to a more favorable user experience.

The development team has also added a universal error page to the web storage interface which users will be rerouted to if an error occurs trying upload or retrieval of a file. Additionally, the team has continued testing the integration of Badger DB into the system and are actively working on solving some issues related to this integration. Further work has been completed on integrating a streaming implementation of file uploads to handle larger files. In conjunction with the broker node performance improvements of saving binary data outside of the SQL database, this streaming implementation will significantly help with scalability.

Finally, the team has been working on in-browser previews for Oyster files, and some small improvements on streamable have been made, most notably download progress can now be tracked and displayed visually.

As always, we encourage our community to post any questions or comments regarding these development updates to our Reddit or Telegram channel.