Stories by Raven Protocol on Medium

‍⬛Ravnest Dev Summary 2024

Raven Protocol — Tue, 21 Jan 2025 11:19:30 GMT

1. Published a Research Paper about our framework Ravnest on Arxiv.

Ravnest: Decentralized Asynchronous Training on Heterogeneous Devices

2. Built and deployed a Readthedocs website for Ravnest that contains extensive documentation, feature updates and usage examples.

https://ravnest.readthedocs.io

3. Developed GRPC enabled communication paradigms and restructured codebase.

4. Integrated data compression techniques to our backend, reducing network overhead and improving training efficiency.

5. Common script for all provider roles for ease of deployment. Ravnest automatically infers the role of the providers within clusters.

6. Support for heterogeneous training jobs allowing both CPU and GPU based systems to participate in same training session.

7. Model compute requirement detection improved and automated in clusterize method.

8. Major improvements to our model splitting technique to extend support for contemporary LLMs.

9. Successfully performed initial benchmarking experiments on small LLMs and vision models.

10. Support for Custom Trainer added, allowing developers to define their own training loops for non-conventional training flows.

11. Introduced new Broker role in Ravnest ecosystem that orchestrates decentralized training jobs.

12. Introduced a new compute mode on top of GRPC that supports PyTorch Distributed training for local LAN/Infiniband connected setups through nccl/gloo backends.

13. Secured GPU systems for benchmarking against state of the art frameworks (like MegatronLM by Nvidia and ColossalAI) through sponsorship programs.

14. Positive technical discussions with players in decentralized compute ecosystem towards potential adoption of Ravnest.

OFFICIAL CHANNELS:
Official Email Address: founders@ravenprotocol.com
Official Website Link: http://www.RavenProtocol.com
Official Announcement Channel: https://t.me/raven_announcements
Official Telegram Group: https://t.me/ravenprotocol
Official Twitter: https://twitter.com/raven_protocol
Official Medium: https://medium.com/ravenprotocol
Official LinkedIn: https://linkedin.com/company/ravenprotocol
Official Github: https://www.github.com/ravenprotocol
Official Substack: https://ravenprotocol.substack.com
Official Discord: https://discord.gg/Njq8QxYUKR

🐦‍⬛🪺Ravnest Dev Summary 2024 🎉 was originally published in RavenProtocol on Medium, where people are continuing the conversation by highlighting and responding to this story.

The Blueprints of Ravnest’s Architecture

Raven Protocol — Tue, 06 Feb 2024 11:21:24 GMT

We are thrilled to share the blueprints of Ravnest’s architecture.

Figure 1 depicts the process of matchmaking, cluster formation and model fragmentation into submodels on an intermediary server. The requester and all available compute nodes will connect to this intermediary server to participate and get the training started.

Figure 2 elucidates upon what goes on inside each cluster, running parallelly. It shows how data is fed through the different nodes as micro batches and asynchronous training is achieved using zero-bubble mode parallelism.

Figure 3 shows an overview of how Global model parameter averaging takes place across a set of clusters in accordance with the parallel multi-ring all reduce method described in our arxiv pre-print. This is triggered periodically to sync up model parameters across clusters.

Link to Research Paper: https://arxiv.org/abs/2401.01728

Github Repository: https://github.com/ravenprotocol/ravnest

— The Raven Protocol Team

The Blueprints of Ravnest’s Architecture was originally published in RavenProtocol on Medium, where people are continuing the conversation by highlighting and responding to this story.

Ravnest: Decentralized Asynchronous Training on Heterogeneous Devices

Raven Protocol — Mon, 05 Feb 2024 10:16:22 GMT

We are happy to share that we’ve published a new research paper!

Check the Cornell University arXiv pre-print for Ravnest, our new decentralized algorithm: https://arxiv.org/abs/2401.01728

We request everyone to have a look at this research paper and give us feedback on any improvements for future iterations.

The implementation is underway and the updates can be found on the following GitHub Repository: https://github.com/ravenprotocol/ravnest

Please note that we are not inviting contributions immediately as we are working on the experimental and comparative analysis of our proposed approach and thereby making regular drastic updates to this repository.

— The Raven Protocol Team

Ravnest: Decentralized Asynchronous Training on Heterogeneous Devices was originally published in RavenProtocol on Medium, where people are continuing the conversation by highlighting and responding to this story.

Concurrent Training of a Requester Graph Across Multiple Participating Providers

Raven Protocol — Tue, 13 Jun 2023 12:49:13 GMT

🚀 Release Notes 🚀

We are excited to announce the next version of Ravenverse which supports concurrent training of a Requester Graph across multiple participating Providers.

Requester Side

1. The number of participants required to compute a graph is now determined and set automatically by our backend based on the complexity of the deep learning graph, operations, weight size and a bunch of other parameters. Requesters can no longer set this number manually in the ravop.execute() function.

2. Example scripts based on fine-tuning of GPT-2 model has been added in the Ravenverse GitHub Repository. The Requesters can first generate a “.pt” (pytorch) model file and there and load it up for distributed training in the Ravenverse. As an example, we have added a Poem Generator GPT which can be trained to write poems based on different emotions like fear, anticipation, mystery, horror etc. Additionally, there’s a Number-Sorting application using GPT that takes in an input vector and simply sorts it.

3. Novel optimisation technique implemented for subgraph formation. Significant speedup observed in graph compilation time.

Provider Side

4. Improved ping-pong refresh rate for more rapid assignment of subgraphs to providers.

5. New subgraph computation mechanism with reduced payload size for more efficient transfer of results. Massive speedup observed.

6. Backup provider support for maintaining graph progress in case of disconnect cases.

7. GPU benchmarking metrics revised based on graph complexity.

Our Libraries:

Ravpy: https://pypi.org/project/ravpy/
Ravop: https://pypi.org/project/ravop/
RavDL: https://pypi.org/project/ravdl/

You can find the documentation for each library in the respective repo Readme files. Please try them out and let us know if you run into any problems.

Raven Protocol GitHub: https://github.com/ravenprotocol

Enjoy the new release! ❤️

— The Raven Protocol Team

Concurrent Training of a Requester Graph Across Multiple Participating Providers was originally published in RavenProtocol on Medium, where people are continuing the conversation by highlighting and responding to this story.

New Release Announcement: GPU Support in Ravenverse

Raven Protocol — Mon, 01 May 2023 12:58:00 GMT

Photo by Nana Dua on Unsplash

🚀🚀 Release Notes 🚀🚀

We are excited to introduce GPU support to Ravenverse! Requesters can now have complex Graphs computed by Providers that can contribute GPU resources.

Requester Side Changes:

Requesters can request for GPU resources by specifying the gpu_required=“yes” parameter in the R.Graph function during Graph definition.
Based on the complexity of the Graph, Ravsock generates minimum system and GPU requirements and only allows Providers meeting those requirements to participate.
Requesters can now directly load PyTorch models into RavDL by using the R.model op. This op takes the path to the TorchScript file of the PyTorch model as a parameter and allows distributed training and inference via the Ravenverse.
Requesters can now persist their entire R.model and retrieve from the Ravenverse post training for further use
Significant speed up in compilation, distribution and computation observed.

RavDL: https://github.com/ravenprotocol/ravdl

Provider Side Changes:

Improvements in connectivity checks
list_graphs functions can be used to check for GPU requirements along with system requirements
Providers can only participate in graphs for which they meet the minimum requirements
Computation speed and efficiency increase observed
Memory Usage optimised

Ravpy: https://github.com/ravenprotocol/ravpy

We have included helper codes in our Ravenverse GitHub Repository for Requesters and Providers.

Ravenverse GitHub: https://github.com/ravenprotocol/ravenverse

You can find the documentation for each library in the respective repo Readme files. Please try them out and let us know if you run into any problems.

Raven Protocol GitHub: https://github.com/ravenprotocol

Enjoy the new release! ❤️

— The Raven Protocol Team

New Release Announcement: GPU Support in Ravenverse was originally published in RavenProtocol on Medium, where people are continuing the conversation by highlighting and responding to this story.

New Release Announcement: Selective Participation

Raven Protocol — Wed, 22 Mar 2023 14:56:20 GMT

We are excited to announce a new iteration of Ravenverse!! In this release, our primary focus has been on Selective Participation.

The previous releases of Ravenverse and its supporting libraries relied on the fact that providers can connect to the network any time they wish and if any ongoing requester graphs are being executed at the moment, they will instantly participate and start receiving computations. However, this approach has some caveats. For instance, we observed that if there are no available pending requester graphs in our network, there is practically no point for providers to keep their systems up and running since they won’t be receiving any computations. Also, it was not guaranteed that all providers in our network will definitely receive computations if one requester were to execute their graph. In order to fix these shortcomings of our network, we are introducing a new paradigm in this release — Selective Participation.

Release Notes

Provider side changes (Ravpy v0.16):
https://pypi.org/project/ravpy/

1. Providers must now manually participate in a specific graph. This means that providers no longer need to keep their systems running and wait indefinitely to receive computations. The list of available graphs for computation can be checked by a provider by running the Provider/list_available_graphs.py in the Ravenverse repository.

2. A graph starts execution when its required number of participants is met. The table will show a bunch of information about each graph including System Requirements, Minimum Stake Required, current Active Participants, and the total number of Required Participants in the graph. Before participating in a graph, Providers must ensure that their system meets the system requirements of that specific graph and they must have enough tokens in their ravenverse account to stake.

3. Along with their earnings, the staked amount will be returned back to the provider once they have successfully computed their share of subgraphs and returned back the computation results. However, A portion of the staked amount will be deducted as a penalty if the Provider disconnects from the network before computing all of their share of subgraphs.

4. Once all computations of a Provider are successfully completed, they will automatically get disconnected from the network and are again free to participate in any other new graph. They can also check their earnings on their ravenverse website dashboard.

5. Quality of life improvements like Upload/Download progress bars.

6. Improvements to the benchmarking procedure by utilizing a dummy torch model.

Requester side changes (Ravop v0.12): https://pypi.org/project/ravop/

7. After the compilation of their graph, requesters will now be shown the cost of the graph and the maximum number of allowable participants (providers).

8. The requester must set a required number of participants in the execute function.

9. Once the graph reaches the execution stage, this number of required participants and currently active participants will be broadcasted to all Providers who wish to participate.

We have included helper codes in our Ravenverse GitHub Repository for Requesters and Providers.

Ravenverse GitHub: https://github.com/ravenprotocol/ravenverse

You can find the documentation for each library in the respective repo Readme files. Please try them out and let us know if you run into any problems.

Raven Protocol GitHub: https://github.com/ravenprotocol

Enjoy the new release! ❤️

— The Raven Protocol Team

New Release Announcement: Selective Participation was originally published in RavenProtocol on Medium, where people are continuing the conversation by highlighting and responding to this story.

New Release Announcement: Introducing the revamped versions of Ravpy, RavDL, Ravop libraries

Raven Protocol — Fri, 10 Feb 2023 14:31:48 GMT

🚀🚀 Announcement! 🚀🚀

We are excited to announce a new release of Ravenverse!

Introducing the revamped versions of Ravpy, RavDL, Ravop libraries.

1. Ravpy (v0.15): https://pypi.org/project/ravpy/
2. RavDL (v0.10): https://pypi.org/project/ravdl/
3. Ravop (v0.11): https://pypi.org/project/ravop/

We have included helper codes in our Ravenverse GitHub Repository for Requesters and Providers.

Ravenverse GitHub: https://github.com/ravenprotocol/ravenverse

You can find the documentation for each library in the respective repo Readme files. Please try them out and let us know if you run into any problems.

Release Notes:

Ravdl (v0.10)

Massive performance improvement in Graph computations and model training, thanks to a revamped approach to the backpropagation algorithm.
Added a collection of new layers to the library along with provision for defining Custom layers.
Apart from Sequential Models, Requesters can now create Functional Custom Models. This will allow them to deploy state of the art complex models (like GPT variants) with ease.
Many new Activation functions added to the library.
Deprecated the use of the old ravdl.v1 API. The new ravdl.v2 API is much more intuitive and powerful to use.
Refactoring changes to the ravdl codebase.
Updated RavDL readme with a detailed tutorial on how to use the new ravdl.v2 API.

Ravpy (v0.15)

Added support for new deep learning, machine learning and math ops.
Optimized model training computations.
The mathematical backend for ravpy has been shifted to Pytorch. This will allow us to leverage the power of GPU acceleration and other features in future releases.
FTP broken-pipe connection issues resolved.
Execution of Backpropagation Algorithm has been optimized for better performance.
Refactoring changes to the ravpy codebase.
Minimized the probability of subgraph failure. Improved the robustness of the system with better reassignment strategies in case of mid-computation internet connectivity issues.

Ravop (v0.11)

Added support for new deep learning, machine learning and math ops.
Requesters will now receive the exact error messages in case their graph fails.
The post-execution results of computed ops can now be fetched as torch.Tensor objects.

Ravsock & Scheduler-Service

Major improvements to the scheduler-service. The scheduler-service now uses a more efficient algorithm to generate and assign subgraphs to workers. This will result in faster execution of subgraphs.
Improved handling of failed and redundant subgraphs. Graphs will fail only after 5 re-attempts at execution.
Optimized payload formation due to which the size of the payload has been significantly reduced.
Error serving features for Requester.
Dynamic cleanup for inessential data.
Dynamic split size for graph splitting based on complexity.
Revamped the communication channel between the scheduler-service and the ravsock server running on multiple worker threads.
Added support for new deep learning, machine learning and math ops.

Raven Protocol GitHub: https://github.com/ravenprotocol

Enjoy the new release! ❤️

— The Raven Protocol Team

New Release Announcement: Introducing the revamped versions of Ravpy, RavDL, Ravop libraries was originally published in RavenProtocol on Medium, where people are continuing the conversation by highlighting and responding to this story.

Launching Ravenverse!

Raven Protocol — Thu, 06 Oct 2022 21:46:52 GMT

Hi everyone,

Finally🥁

We are thrilled to announce the public release of Ravenverse.

Raven Protocol is decentralizing computing power and creating a requester-provider economy. And Ravenverse is a significant step forward for Raven Protocol.

As a team, we are constantly evolving and improving. Our team has worked hard on this release and will keep improving Ravenverse. Requesters can now develop operations and models, and providers can join in to offer their idle computing resources in exchange for Raven tokens.

Instructions to use Ravenverse:

Create your Ravenverse account and generate an access token: https://ravenverse.ai (Need Metamask)
Clone Ravenverse repository: https://github.com/ravenprotocol/ravenverse
Put your access token and server credentials in the .env file:
Ravenverse Server Url: http://server.ravenverse.ai
Ravenverse FTP Host/Url: http://server.ravenverse.ai
Create operations and models, or join in to offer your idle computing resources

Enjoy Ravenverse! Please let us know if you run into any problems.
Note: We are working on a no-code UI for requesters and providers.

Launching Ravenverse! was originally published in RavenProtocol on Medium, where people are continuing the conversation by highlighting and responding to this story.

Proof of Steak: Latest Rewards Distributed and Beacon Chain Pause

Raven Protocol — Mon, 01 Aug 2022 20:36:10 GMT

Dear Raven Protocol Community,

The June 2022 Staking Period was completed and it was a success! We would like to give a shout out to Samüs from SpartanProtocol.org. He is one of the few developers in the world still ensuring that all projects building on BNB Chain still have the tools they need to automate tasks on-chain. Thank you!

Total Stakers: 259
Total Staked: 1,234,488,590 RAVEN
Total Rewarded: 12,344,883 RAVEN
Circulating Supply: ~3B RAVEN

Starting from August 1, 2022, we will pause BEP-2 (Beacon Chain) staking and start to explore BEP-20 (Binance Smart Chain). This is due to the latest BEP-151 proposal to decomission Binance DEX. While Beacon Chain will be the staking and governance layer for BSC, the upcoming Ravenverse will focus on EVM compatibility and thus our decision is to pause BEP-2 staking.

Motivation from the Proposal

BNB Beacon Chain(BC) is a blockchain developed by the BNB Chain community that implements a vision of a decentralized exchange (DEX) for digital assets. The heart of Beacon Chain is a highly performant matching engine built on distributed consensus that aims to replicate the < 1 second trading efficiency of current centralized exchanges.

Since BNB Smart Chain’s(BSC) launch to mainnet in 2020, the AMM based decentralized exchanges gain great success on BSC which even steal the thunder of order-book based DEX on BC. BC and BSC is a dual chain structure, with the evolution of this structure, BC plays more like a Beacon Chain that helps to enhance the security of BSC as a staking and governance layer. It is not suitable for a Beacon Chain to sustain a high-performance DEX any longer, especially when there is another DEX with massive adoption in the same ecosystem.

Decommissioning the DEX module will reduce the need for system resources significantly for both validators and full nodes. The demand for liquidation can be fulfilled by BNB Smart Chain or the upcoming zkBAS.

zkBAS looks interesting 👀

— The Raven Protocol Team

Raven Protocol White Paper:
https://drive.google.com/file/d/1FAaVKkg_CjxMj-n1yHZc6ufcVDtOU1Ct/view?usp=sharing

Proof of Steak: Latest Rewards Distributed and Beacon Chain Pause was originally published in RavenProtocol on Medium, where people are continuing the conversation by highlighting and responding to this story.

Proof of Steak: September 2021 Rewards Distributed

Raven Protocol — Mon, 01 Nov 2021 00:43:32 GMT

Dear Raven Protocol Community,

The September 2021 Staking Period was completed and it was a success!

Total Stakers: 250
Total Staked: 1,205,740,998 RAVEN
Total Rewarded: 12,057,406 RAVEN
Circulating Supply: ~3B RAVEN

Starting from the July 2021 Staking Period, the staking reward will be 1% monthly. We announced a partnership with Ocean Protocol in May. Raven will become a Compute Provider on Ocean Market, publishing a range of algorithms from Machine Learning to Federated Analytics and Federated Learning. We’re excited to explore what staking could look like in this environment. Please share with us any ideas you have!

— The Raven Protocol Team

Raven Protocol: Q2 2019 Tech and Community Update:
https://medium.com/ravenprotocol/q2-2019-tech-and-community-update-4f836a9a1e97

Raven Protocol: Q3 2019 Tech project development Update:
https://medium.com/ravenprotocol/tldr-raven-stayed-heads-down-building-in-q3-2019-ae5f242dc15d

Raven Protocol: Q4 2019 Tech project development Update:
https://medium.com/ravenprotocol/happy-2020-an-important-update-from-the-raven-team-67b5e88e1b1d

Raven Protocol: Q1 2020 Tech project development Update:
https://medium.com/ravenprotocol/q1-2020-success-during-hard-times-3dcdbb0faba8

Raven Protocol: Q2 2020 Development and Community Update:
https://medium.com/ravenprotocol/q2-2020-the-raven-distribution-framework-solves-the-deep-learning-problem-that-mit-researchers-bc506816ab67

Raven Protocol: Q3 2020 Development and Community Update:
https://medium.com/ravenprotocol/q3-2020-algos-staking-and-burning-51cc57cb7ba

Raven Protocol: Q4 2020 Development and Community Update:
https://medium.com/ravenprotocol/q4-2020-the-raven-distribution-framework-saw-major-strengthening-of-the-ravop-library-ca201e38275a

Raven Protocol: Q1 2021 Development and Community Update:
https://medium.com/ravenprotocol/q1-2021-blockchain-tooling-and-decentralized-web-infrastructure-1f0d498c2b38

Raven Protocol: Q2 2021 Development and Community Update:
https://medium.com/ravenprotocol/q2-2021-heads-down-building-306d4ada6b9d

Raven Protocol Project Review:
https://cryptocalibur.com/portfolio-item/raven-protocol-review

Raven Protocol White Paper:
https://drive.google.com/file/d/1FAaVKkg_CjxMj-n1yHZc6ufcVDtOU1Ct/view?usp=sharing

Proof of Steak: September 2021 Rewards Distributed was originally published in RavenProtocol on Medium, where people are continuing the conversation by highlighting and responding to this story.

Stories by Raven Protocol on Medium

‍⬛Ravnest Dev Summary 2024

The Blueprints of Ravnest’s Architecture

Ravnest: Decentralized Asynchronous Training on Heterogeneous Devices

Concurrent Training of a Requester Graph Across Multiple Participating Providers

🚀 Release Notes 🚀

Requester Side

Provider Side

Our Libraries:

New Release Announcement: GPU Support in Ravenverse

Requester Side Changes:

Provider Side Changes:

New Release Announcement: Selective Participation

Release Notes

Provider side changes (Ravpy v0.16): https://pypi.org/project/ravpy/

Requester side changes (Ravop v0.12): https://pypi.org/project/ravop/

New Release Announcement: Introducing the revamped versions of Ravpy, RavDL, Ravop libraries

Release Notes:

Ravdl (v0.10)

Ravpy (v0.15)

Ravop (v0.11)

Ravsock & Scheduler-Service

Launching Ravenverse!

Instructions to use Ravenverse:

Proof of Steak: Latest Rewards Distributed and Beacon Chain Pause

Proof of Steak: September 2021 Rewards Distributed

Provider side changes (Ravpy v0.16):
https://pypi.org/project/ravpy/