Of all the areas in AI, Multimodal Deep Learning is the one I’m most excited about, especially with Large Language Models (LLMs) like ChatGPT taking the world by storm.
The problem with OpenAI though, is that they are a centralized entity gatekeeping their machine learning resources, only releasing an API for people to integrate ChatGPT - instead of open sourcing their models.
Regardless, hundreds of online businesses can essentially be built now, simply by utilizing an OpenAI API Key. Imagine how many more new doors can be opened, if individuals & companies had access to an open-source protocol, with multi-modal frameworks of machine intelligence.
What is Bittensor?
Bittensor has created the first open-source protocol that is essentially a decentralized neural network & cognitive economy that commoditizes machine intelligence, with the goal of training multi-modal frameworks (text, images, audio, video, etc.) within a blockchain infrastructure and rewarding performance with a custom digital currency ($TAO).
Bittensor relies on miners that fine-tune models to train the network, as well as validators that evaluate intelligence produced by these miners and record it on a blockchain.
It is an open-source machine like Ethereum, but instead of relying on 100s of API connections that constantly need to get updated, you just plug into Bittensor’s meta-model, with your requests being routed to the endpoints that serve them best.
Valuable Machine Learning Concepts
To see a lot of the value proposition, it’s necessary to familiarize yourself with 2 core machine learning concepts!
Mixture of Experts (MoE): Imagine you have a group of friends who are experts in different fields, like cooking, sports, and music. When you need advice on a topic, you would consult the friend who knows most about it. MoE operates on a similar principle.
In a MoE model, each expert is designed to handle specific types of data or situations. The model also has a “gating network,” which is like a coordinator that determines which expert should be consulted for a given input. The gating network assigns a weight to each expert based on their relevance to the input, and the final output is a combination of the experts’ predictions, weighted by their assigned relevance.
Distillation: Think of it like making a cup of strong, concentrated coffee from a large pot of regular coffee. The idea of distillation is to extract the essential knowledge from a larger, more complex model (called the “teacher model”) and transfer it to a smaller, simpler model (called the “student model”).
The process involves training the student model to mimic the behavior of the teacher model by replicating its predictions or outputs. During training, the student model is exposed to the same data as the teacher model, but instead of learning directly from the raw data, it learns from the teacher model’s outputs. By learning from the teacher model, the student model acquires the essential knowledge and can make similar predictions, but with the advantage of being smaller and faster.
Bittensor is a p2p Mixture of Experts model on which individual peers are Distilling knowledge from each other
Bittensor, uses Bitcoins substrate of incentivized compute, to scale the protocol, but instead of rewarding non-productive hashes (BTC), it rewards the creation of machine intelligence under high demand relative to its supply. Bittensor adopts the same mechanism that scaled Bitcoin to the largest supercomputer in the world too, by hashing power to scale a decentralized mixture of expert’s model or to conceptualize — a neural network of neural networks.
Bittensor’s Utility Token — $TAO
The custom digital currency $TAO is the ownership, governance, and value transfer layer of the Bittensor network.
$TAO represents weight in the network as it allows to govern the incentive mechanism that all miners work for, models (miners) will align their produced intelligence to your demand if your stake is high enough. the more stake the higher priority miners will give serving your requests. This creates an open market for machine intelligence
The value of $TAO is connected to the value of the intelligence that is produced by the network. Ownership of the network comes with holding $TAO. So, if someone holds a fair amount of $TAO, he gets to decide, on what the network will be trained on (there are mechanisms against collusion in place).
If an external company wants to use Bittensor, for example to develop a specific software, it needs $TAO. In order to have the request bandwidth to make the application worthwhile, a certain amount of $TAO is needed. Also, the miners and validators of the network are incentivized with $TAO.
$TAO tokenomics are inspired by Bitcoin: The max supply of $TAO is 21 million tokens, and every four years, there is a halving event (the first one in 2025). The current issued supply is around 4 million $TAO and that’s all been created by being mined like $BTC.
