Raven Protocol: 2018 Achievements
What a crazy ride 2018 was. The markets were down ~90%+ from all-time highs in January. Many strong projects shut down. Many established companies gave up. We don’t blame them. It’s hard to survive when teams held on to mostly crypto.
Only the strongest companies, building real tech, with real user adoption will come out of this bear market alive.
Raven Protocol provides cost-efficient and faster training of deep neural networks. We are creating a network of compute nodes that utilize idle compute power for the purposes of AI training where speed is the key.
Why was Raven able to survive 2018?
1. We are solving a problem we had ourselves
Raven Protocol was born out of the founders’ own needs. In the picture below from 2017, Sherman Lee and Rahul Vishwakarma look happy. But they were super stressed out during this period.
They were running their respective AI startups and user growth was coming fast. Costs exploded and on AI training alone, they were being invoiced $5–10K/month each from AWS. And the training times were slow. Two weeks to train a 1M image dataset was not acceptable.
2. We didn’t listen to AI experts at Google or Facebook
When we set out to solve the AI training problem, we had no idea if our approach would actually work. So we talked to machine learning engineers, data scientists, and AI researchers at corporations who have spent billions of dollars on AI.
The brightest minds in AI at Google and Facebook did not think our approach was feasible. Even if we were able to build out the compute nodes in our network, they were skeptical that the distribution of computations would actually make anything cheaper or faster.
We really questioned ourselves at this point. These are people at the top of their game with all the resources in the world to do AI training.
But there was a key difference that made us not listen to them. It was because they had all the resources in the world. AI engineers at Google and Facebook do not have a unique insight into the problem like us.
3. We built a proof of concept
We could have written a whitepaper and launched an ICO, but the first line of anything we wrote was a line of code. We proved to ourselves that our unique approach would work.
Here’s a screenshot of when we fired up the code and connected 14 compute nodes to the local Raven network:
4. We discovered our unique value proposition
Raven performs AI training where speed is the key.
Raven is the answer to ever increasing compute demand for Deep Learning. We developed a completely new approach to distribution that speeds up a training run of 1M images from two weeks down to a few hours.
Existing deep learning distribution methods and frameworks have come a long way, no doubt about that. However, it still needs gargantuan servers for the speeds at which demand is increasing for AI. Data and model parallelism have been used to optimize this, but it creates inherent latency in the network and is not scalable beyond a certain limit.
We solve latency by chunking the data into really small pieces (bytes), maintaining its identity, and then distributing it across the host of devices with a call to action: gradient calculations.
5. We are building a framework for AI training from the ground up
Our proof of concept worked in a local dev environment. To get this to work in production with external compute devices would require extremely hard work. Thus, we had to start building our own framework.
We couldn’t simply build upon one that existed as our approach was unique. Other existing frameworks have a different approach for calculation and data distributions which couldn’t have solved our purpose. So we had to re-write the foundation layer which could support minuscule distribution over a distributed setup.
More on Dynamic Graph Computation in our framework is on a separate blog post, but here’s a quick overview:
Dynamic Graph Computation
All the frameworks operate on tensors and are built on the computational graph as a Directed Acyclic Graph. In most of the current and popular deep learning frameworks including Tensorflow (before Eager Execution), the computational graph is static in nature. However, frameworks like PyTorch is dynamic, giving a lot more options to researchers and developers to fiddle around with their creativity and imagination.
A major difference between static and dynamic computation graph is that in the former, the model optimization method is preset, and the data substitutes the placeholder tensors. Whereas, in the latter the nodes in a network are executed without a need for any placeholder tensors. Dynamic computation holds a very distinguishable advantage in cases like language modelling where, the shape of the tensors are variable during the course of the training.The benefit of a dynamic graph is its concurrency, and it is robust enough to handle the contributor addition or deletion, making the whole Raven training sustainable.
Raven is thus capable of eliminating the latency and scalability issues, with both the approaches. Hence, distributing the training of any deeper neural network and their larger datasets, by getting rid of the added dependency on the Model replication. Data is also sharded in smaller snippets.
In fact, the Model is intact at the Master Node, and the heavy lifting is distributed in the tiniest snippets of data subsets over the network of contributors. The resultant gradients, after the calculations that happen at the node/contributor ends, are sent back to the Master Node.
This creates a ton of difference, as it is easier for calculations to pass through from machine to machine, rather than creating multiple replicas of a complicated Model.
6. We aligned one of our cultural values with being a revenue generating business
The majority of projects out there have no business model other than an increasing price in their token. We get it. Making money is hard. But if we want to drive this entire industry forward, we need to make it sustainable long term.
Raven is doing our small part. We have a clear path to revenue from day one of launch. The team started demoing our prototype to AI companies. Over 30 of them committed to being beta customers on the spot. If they understood the problem deeply, they would shout about our mission from the mountain tops. Industry leaders supporting us means the world.
Volareo, a team who has shipped over 500,000 hardware units globally, will use us to power the NLP in their smart speakers.
Why Raven Needs to Exist by Dr. Sebastian H.R. Wurst
Project Overview by Cryptonauts
Gagarin ICO excited about our project
Metacert, the team behind Cryptonite protecting millions of users in crypto, gave Raven a cameo ;)
CryptoRushMovie.com came to Hong Kong to include Raven in their film. To be released in 2019!
7. We raised a seed round during the hardest time in crypto
Our local development environment was great for our own personal use. We got to train models quickly and cheaply (the cost of electricity and the sunk cost of devices we already owned).
However, we needed to put this power into the hands of the AI community and bring on more compute devices to the network. Incentivizing nodes to do that was a perfect fit for our decentralized and distributed training protocol.
In Sep 2018, many of the beta commitments further showed their support by offering to pre-pay. That was enough momentum to kick off our seed round and we closed it fast. People said it would hard and damn near impossible to raise in a deep bear market. But the Raven team executed flawlessly. We are lean and we know how to do more with less money.
Here’s a special shout out to early backers who will not rest until they see Raven in production. We thank you for being on the journey with us!
There are many more founders, ecosystem builders, and funds who participated in the round as well. More of them will be announced in a later update on why strategic partners matter to us. We’re in this for the long-haul and we are only allowing people who are aligned with that to get involved.
