Decentralized Hybrid Neural Networks
“Thinking Together” using Dopamine.ai
When I was a young programmer, I was sometimes asked what I did for a living when I wasn’t quite in the mood to have a lengthy discussion about software development. In such cases, I could usually get away with saying one word: “computers.” Back then, no one asked whether I was a software or hardware engineer, not to mention front-end or server-side, data scientist or QA, IT or RT: we were all simply “computing-related.” But today, if I were to forget what age we now live in and try to get away with using the same approach — I’d come off as being rude or dismissive. By analyzing the evolution of computing related workforce, we can extrapolate the future of software market structures.
In the lines below, we show how the future software market might look, focusing on AI-oriented software, and how new decentralized platforms, like Dopamine.ai, will help it flourish.
From Viruses to Mammals
The situation I described above fits the statement below:
In growing technical fields, tomorrow’s jobs will require a much more specialized skill set than that of today’s professions.
Now, it would be reasonable to say that many of human professions are expected to be replaced by AI modules, or AI-related knowledge. And given that AI is a growing field, we can also say that:
Tomorrow’s AI modules are expected to be much more granular than today’s.
A different view getting to a similar conclusion would be in the following puzzle. By which defining characteristic are the following organism types ordered: Virus, Bacteria, Fungus, Nematode, Insect, Mammal?
Most of you would probably say “by the level of sophistication,” while others would say “by genome size.” The truth is that both answers are correct. Usually, as solutions to problems get more advanced, they become more complicated and harder to determine. We see the same thing with advances in the AI landscape in recent years: architectures get more profound and more complicated. And we expect to see this tendency continue. So, again:
Tomorrow’s AI modules are expected to be much more granular and specialized than today’s.
Eyes of the Hawk, Ears of the Wolf, Strength of a Bear and Speed of a Puma
So if each one of the above AI modules specializes in its own domain, in a decentralized economy, we would expect a different entity to own each module. Each entity would then continuously optimize and evolve these modules. Most of the engineers among you would argue that combining modules that were not built together (e.g., with the eyes of a hawk and the ears of a wolf) would create a paralyzed monster, which is true in regular systems. But in some types of AI, like Neural Networks, the plasticity of such systems makes it much easier to glue together less-related modules. This capability would be impossible in older, stricter software systems.
Let’s take as a simplified example the following case: consumer A wishes to have an image classifier for digits (to all data scientists out there, my apologies. I know this classification can be done in a few lines of code, so this is only an example) and let’s assume there are three relevant AI modules out there in the ecosystem that were used in other projects:
- A Retina module — specializing in efficiently extracting low-level features from images, owned by Entity “R.”
- A Visual Cortex module — specializing in efficiently extracting high-level features from low-level modules, owned by Entity “V.”
- A Classifier module — specializing in providing good estimates about which class each sample belongs to, owned by Entity “C.”
Each one of the above modules is assumed to be developed, optimized, and maintained by its owning entity, as represented below:
Through the Dopamine network, the owning entities allow their modules to be instantiated, without revealing their internals, and trained for specific cases, in exchange for rewards. For example, entity C’s module could be used for other classification tasks, not only visual (like a voice mood detector):
The other modules are equally flexible. For example, the “Retina” module could have been used in the same way as the module C, to process images in various image classification tasks (like face recognition), and the”visual cortex” module could have been used in other tasks that would require high-level feature processing, like object detection (of road signs):
Again, my apologies to our neurophysiologist readers, the above names are only for clarifying the specialty of each module.
Below we illustrate how, using Dopamine, client “A” can create a “hybrid being” that uses a retina of “R,” the visual cortex of “V” and the classifier of “C”, and puts that new hybrid being to a specialized task.
Coding the Hybrid
In our Github, you can find the code for each one of the entities above. The code is quite simple, and mainly describes the rules and rewards every such provider expects in exchange for using its module. However, if you have not read our previous examples, like the hello-world example, we recommend starting there. Note that the vision is that each one of these modules would have a certain level of complexity and pre-optimization, which is not applied in our example.
The code includes a simplified program for entities R, V and C. The interesting part of this code is that the client entity A, can create a similar structure to the one below and instantiate and train a decentralized entity (blue in diagram below), all while rewarding the relevant participants via a smart contract:
Surprisingly, the client side code is much simpler than we initially expected; the consumer only needs to tell the program that she expects the data to flow in a pipeline created by these three entities, and state the reward she is willing to pay for that pipeline. Under the hood, the Dopamine network takes care of:
- Reward negotiations
- Pipeline creation
- Matching validations
- Efficient data flow
- Reward book-keeping
- And more…
We demonstrate that the “hybrid being” created by the client is able to learn:
The new hybrid being is actually a decentralized neural network, which lives on the Dopamine network and rewards all of its creators according to pre-agreed smart contracts. For example, whenever entity “B” (red below) asks to use this new being for classifications, it would need to reward all relevant entities, including entity A, according to the predefined smart contract:
Regulating the Hybrid
Some of you might argue that the ability to develop AI with no single point of control is too risky. My argument is that such a structure is the most regulable possible. In this type of AI structure, no central authority has all the knowledge to build risky AI based solutions. The AI is global, and it is quite easy to sanction bad players. This new AI represents a marketplace where you need the cooperation of many players to achieve advanced results. For example, if a large number of players agree to obey a certain AI treaty, players excluded from that treaty would find it very difficult to develop any solution without joining the treaty.
Conclusion
The above example shows real code for the creation of a decentralized AI (the blue in the above diagram). This AI is decentralized in the sense that no one controls it, it can be built with no single point of failure, and that many players contributed to its creation and optimization. We see the potential of such structures not only for the sake of decentralization, but as a means for democratizing AI and data, efficiently developing new AI solutions, and having better control over AI. For more updates regarding Dopamine network, please subscribe here.
