CryptoRAT litepaper: serverless Web3 future of AI infrastructure

Introduction

CryptoRAT is a serverless platform for the fast, scalable, and cost-effective deployment of GPU-accelerated AI models. It’s a simple tool for data scientists to deploy and service AI models via infinitely scalable APIs. We provide this service by operating a peer-to-peer network of compute nodes that we manage with our task dispatch software.

Anyone with a consumer or data center device on Windows or Linux with a high-end NVIDIA GPU can host a node and provide compute power to data scientists and earn money. Our hosted APIs are scalable and reliable thanks to CryptoRAT not routing the tasks directly, but managing connections on a peer-to-peer network. Once the connection is established through our dispatch software, clients and hardware providers interact without any intermediaries.

Typically, a cloud GPU server costs 5–20 times more than what it could potentially earn by mining crypto: 5x for smaller local clouds, and 20x when it comes to Google or AWS. We source cheaper GPU power by giving out gamified rewards to hosts of our compute nodes. Becoming a host is easy, as our node app is effortless to install. This way we make the arrangement beneficial for both scientists and GPU owners.

We’re laser-focused on making our software extremely efficient in one specific task — serving trained models. By focusing on this, we are able to obtain sustainable power from consumer GPUs despite their individually unpredictable uptimes.

Our short-term goal is to democratize AI deployment by:

• Simplifying DevOps
• Slashing cloud expenses

Our mission is to create a solution for AI app deployment that is ideal for both Web3 and Web2 businesses. Web3 projects will feel right at home, because our service will be:

• Crypto-native
• Decentralized
• Scalable

We believe that our insights into the challenges of the AI market will allow us to satisfy the demand of a significant number of businesses, and that our solution will eventually become the universal protocol for AI in Web3.

Сhallenges of AI development

The hardware is way too expensive

Not only are AI solutions hard and costly to develop, they also require a fair bit of operational expenses in order to be deployed and effectively utilized. One side to this problem is the usual need for significant hardware resources, GPUs being the go-to option.

AI is on its way to becoming ubiquitous, and the demand for computing resources has followed suit: the total available market for cloud AI inference was already almost $40B in late 2020 and it’s estimated to exceed $70B by 2025. On a more practical level,hosting a single AI model in a cloud can easily cost $400 a month for just the GPU, and the overwhelming majority of AI service providers have numerous models deployed. There are other significant expenses as well, which can make the entrance threshold to this market quite steep.

Deploying AI apps is way too complicated

Another significant obstacle in getting an AI model off the ground is all the work required to do so. Oftentimes, just setting up an AI model requires intricate and resource-intensive work, work that usually can only be done by experienced machine learning engineers who are expensive to hire.

Because getting started is so expensive, progress in AI development has suffered. Smaller players that could contribute to the field — startups and mid-sized businesses — often find themselves struggling with this additional barrier.

Creating AI-driven Web3 apps is way too hard

The market for AI-driven Web3 apps is just starting to take shape, so naturally there’s a long way to go. To be truly Web3, an app needs a decentralized GPU cloud that is crypto-native, trustless, and functions via smart contracts.

All in all, Web3 is nowhere near this stage yet. There are simply no services like this as of now, which has created a vacuum waiting to be filled.

CryptoRAT solutions

CryptoRAT cloud is here to make serving AI models in production elementary. Our peer-to-peer network allows hosted AI models to provide near-real-time processing of large data streams in a simpler and more affordable way than that of the traditional cloud. While our current focus is traditional Web2.0 customers, CryptoRAT cloud will eventually evolve into a protocol for anyone to use and leverage to build diverse client-side software.

We source cost-efficient hardware resources

80% of the available computing power in the world, both in data centers and PCs, is sitting idle or is used for mining. We’re introducing AI scientists to all that hardware. Our Hardware Provider app is cross-platform and needs only limited access to device resources: when active, it takes up 80–90% of a GPU, leaving the rest available for other needs. This expands the available hardware supply pool to include gamers, who can still use their PCs for lighter tasks while earning, and data centers looking for higher utilization on demand.

We simplify model deployment

The secret ingredient of a solution like ours is its usability for B2B. In our case, it’s our software development kit (SDK) — the interface between data scientists and CryptoRAT. It’s simple, streamlined, and the pricing model is transparent and doesn’t depend on cryptocurrencies.

We scale via Web3 technologies

Crowdsourced hardware is inconsistent and hard to count on, so our PoS blockchain-inspired architecture tackles it by turning each hardware provider into a tiny DHT node, which can be quickly found when available. We automatically establish a p2p connection between AI developer servers and compute nodes without any type of task processing on our side — we only make sure that each client is always connected to enough supply. This architecture enables almost infinite scalability, as there are no centralized bottlenecks, which paves the way for a truly decentralized Web3 protocol.

CryptoRAT connects a client to a compute node

B2B customer side

Usage

The primary interface of the CryptoRAT protocol is an SDK focused on processing computer vision tasks.

The SDK is here to bring:

• Ease-of-use
• Capability to process huge data streams
• Affordable and transparent pay-as-you-go pricing

A data scientist equipped with our SDK will be able to:

• Upload ONNX-compatible models to our storage
• Deploy an API to serve any uploaded model
• Send tasks to the API and retrieve results
• Check the cloud credit balance and the SDK key status

The SDK software will automatically establish and manage the p2p connection to compute nodes, allowing clients to utilize the compute network in the same manner as they would use a regular asynchronous API. This will simplify AI model deployment down to just a few lines of code.

Pricing

The CryptoRAT pricing structure is simple and straightforward. There is no fine print or hidden fees. Our goal in terms of prices is to cut both costs and complexity in relation to what Google and AWS offer.

Two pricing models are available:

• Pay per instance: best for hosting an API and keeping it online at all times.
• Pay per request: suited for one-time bulk processing tasks, demos, temporary solutions, or apps with unpredictable loads.

Pay per instance

This option is for hosting a single always-on model and interacting with it via an API. The limits are as follows:

• Model file size
• Input size
• Guaranteed requests per second
• Model update frequency
• Key lifetime

Pay per request

This option is for hosting any number of models and applying them on demand. The only limits are:

• Model file size
• Input size

This plan also comes with a fixed number of virtual credits which you can spend on sending requests to the model. The credit cost of a single request depends on input and model size.

Hardware Provider side

Being decentralized, infinitely scalable, and relatively cheap calls for a massive number of GPUs. With our Hardware Provider app, anyone with a supported NVIDIA GPU will be able to share their compute power with the network and earn while doing so.

The app will be available for Windows and Linux OS. We’re building it with ease-of-use as our top priority: the only actions required to start earning will be logging in and pressing the “START” button. Meanwhile, under the hood the app will check the hardware, deploy the compute node, connect it to our dispatch software and get started processing AI or performing other tasks like mining, in case of low demand or hardware oversupply.

Rewards for providing hardware

The earnings for hardware providers are structured to attract gamers and retail users in a way that:

1. Is simple and controllable for a user
2. Incentivizes behavior which is beneficial to the network
3. Automatically balances network hardware supply with demand

Each provider will earn USD for their work and CRAT (CryptoRAT reputation tokens) for beneficial behavior.

USD rewards

USD rewards will be distributed in rounds. To earn a reward, the user will need to provide GPU resources for the whole round (currently a round lasts for 1 hour). The total amount of USD consists of a guaranteed basic reward and a bonus.

Guaranteed rewards will be distributed to any network participant who commits their hardware for the duration of a whole round. This reward will be equal to the average mining income for a given GPU.

Bonus rewards will be shares of the network income generated from B2B customers. A user’s share will be calculated according to strength of hardware and reputation.

In order to simplify the system, a power rating will be assigned to each GPU provided. This rating is discrete and ranks GPUs by their performance: powerful GPUs are rated higher than lower-grade ones. Example: a low-grade GTX 1050 is rated 1, while a monster GPU like a 3090Ti gets a rating of 5.

At the start of each round, participants will be ranked by their GPU power rating multiplied by the amount of CRAT owned. In case of a tie, we will take the total provided uptime and registration date into account. Network income will be distributed according to rank: the higher the rank, the higher the income share. The number of ranks receiving a non-zero bonus will be limited and defined by the total network income generated by customers.

The total reward will be counted in USD and be paid out in any token on the Everscale blockchain.

CryptoRAT reputation tokens

CRAT will be our very own utility token on the Everscale blockchain. Network participants will be able to use them to compete in USD rewards distribution. The tokens will serve more purposes in the future, for example burning tokens to get permanent bonuses, voting in the DAO, etc.

CRAT tokens will be given to hardware providers whose actions benefit the network: providing long streaks of uninterrupted uptime over multiple rounds, providing uptime during peak hours, and correctly validating the computations of other network participants. Users will also be able to get them via airdrops and promotional activities or buy them on the market.

An example of reward allocation

Here’s a simplified example of the reward distribution algorithm with a $10 network income. The leaderboard will be automatically calculated and published at the beginning of each round.

Simplified example of reward distribution

Reward sums here are placeholders and don’t reflect real distributed bonuses. Actual bonuses will change frequently, because they depend on the extent to which businesses use our network for their AI model inference, as well as on the funding available for airdrops and other promotional activities.

Team

The CryptoRAT team is committed to building a truly reliable and useful service. As a team, we have the skills and expertise to take this project to where it needs to go. We believe that we are uniquely equipped to overcome the challenges in this field.

Igor Rekun

An AI expert with a deep understanding of high-performance computing, Igor is passionate about making sure that our technology (written in Rust and powered by NVIDIA TensorRT) works at speeds never seen before. After writing his first custom GPU driver in high school, he went on to co-found three AI startups, including Optia.ai and Handl.ai.

Andrey Kiselev

Andrey is a skilled manager of tech projects, currently utilizing his business network and technical expertise to make sure we are giving our customers something they’ll love. His Data Science skill set, combined with years of management experience at Handl.ai, neuro-core.ru, and, currently, tech-giant Yandex, will ensure the smooth and timely development of CryptoRAT.

Dimitry Lesnevsky

With a passion for applying modern tech and a data-driven approach to each channel of digital marketing, Dimitry has managed over $10M of marketing funds for worldwide brands such as Under Armour, PayPal and Royal Canin with stunning efficiency. He later co-founded an AdTech company, Optia.ai, and the Vinou advertising agency. His marketing management skills are going to power CryptoRAT distribution.

Further info and updates

We expect our path to a web3 serverless future to be nothing short of fascinating. We’d love to keep you up to date on the technical details, which are bound to be unique and exciting, as well as on CryptoRAT as a whole. Join us on our journey:

• Follow us here on Medium for major updates
• Sign up for emails to partake in beta testing
• Get in touch with anyone from our team on LinkedIn
• Drop a line at rat@cryptorat.army for further inquiries.