The Blueprints of Ravnest’s Architecture

We are thrilled to share the blueprints of Ravnest’s architecture.

Figure 1 depicts the process of matchmaking, cluster formation and model fragmentation into submodels on an intermediary server. The requester and all available compute nodes will connect to this intermediary server to participate and get the training started.

Figure 2 elucidates upon what goes on inside each cluster, running parallelly. It shows how data is fed through the different nodes as micro batches and asynchronous training is achieved using zero-bubble mode parallelism.

Figure 3 shows an overview of how Global model parameter averaging takes place across a set of clusters in accordance with the parallel multi-ring all reduce method described in our arxiv pre-print. This is triggered periodically to sync up model parameters across clusters.

Link to Research Paper: https://arxiv.org/abs/2401.01728

Github Repository: https://github.com/ravenprotocol/ravnest

— The Raven Protocol Team

OFFICIAL CHANNELS:
Official Email Address: founders@ravenprotocol.com
Official Website Link: http://www.RavenProtocol.com
Official Announcement Channel: https://t.me/raven_announcements
Official Telegram Group: https://t.me/ravenprotocol
Official Twitter: https://twitter.com/raven_protocol
Official Medium: https://medium.com/ravenprotocol
Official LinkedIn: https://linkedin.com/company/ravenprotocol
Official Github: https://www.github.com/ravenprotocol
Official Substack: https://ravenprotocol.substack.com
Official Discord: https://discord.gg/Njq8QxYUKR