Unlocking the Hidden Potential of Existing Open Source Code
The open source revolution has done wonders for AI research, alongside its dramatic impact on more workaday domains like operating systems for servers and mobile devices (where OSS Linux and Android are dominant) and embedded devices (Linux once again). These days, if you develop a groundbreaking new AI algorithm, it’s no longer so critical that you publish this algorithm in a famous journal or conference (though this may be worthwhile still if you’re an academic, or working for some particular industry research labs). Rather, you can post a description of your algorithm online for free (e.g. submitting it to Arxiv.org), and put your code in GitHub or some other online repository where anyone can check it out and use it, and improve it if they want to.
This is great for researchers, but all this brilliant open source AI code sitting in online code repositories doesn’t do typical commercial software developers much good. Even if a commercial developer has a need that is fulfilled by some piece of available open source code, most commercial developers don’t have the know-how to figure out how to use a piece of research code and integrate it into their application. Further, it’s not always easy to tell which pieces of code available online are high quality in which ways. One can have code that is brilliant algorithmically but not scalable. Or code that is very fast but makes too many internal approximations, so it falls apart when trying to solve complex problems in it’s given domain. Or code that only actually works on the examples in its associated tests and research paper. Or code that gives great results when applied to a suitably large and diverse dataset, but poor results when applied to small datasets. Etc. AI is complicated and there are many possible cases. The result is that commercial AI users who are not AI developers themselves, will often recourse to using commercially available AI services for a high fee, instead of the open source code freely available. Or if they do use free open source code, they gravitate toward the simplest possible approaches, or toward whatever has recently been the trendiest.
From the AI researcher’s perspective, one consequence of this situation is that most open source AI code, even if it’s really good and solves important and relevant problems, won’t get used hardly at all. Because few people have the time and/or expertise to figure it out. This is a problem in terms of bringing revenue to AI developers for their work, and also a problem simply in terms of the intelligence of the overall software ecosystem. More software would be more intelligent, if the reservoir of intelligence comprised by existing open source AI code could be better leveraged.
Recently, some major companies have ventured into the open source AI space, and have released fairly slick toolkits, together with well-documented APIs and associated tutorials, and elegant visualizations and user interfaces. These initiatives make certain AI modules easy to integrate, even for software application developers with only a limited understanding of AI. However, they are limited to narrow classes of AI techniques — and in some cases one might argue they serve a counterproductive role, by directing AI developers and users with excessive frequency toward these particular classes of techniques. I.e. many people end up using the AI approach that comes with the slickest toolkit, rather than the one that suits the problem and will give the best results in the given context.
SingularityNET is designed to unlock the potential of the existing open source AI code that’s sitting out there, and the new AI code to be open sourced day by day as the global R&D community progresses. What if, instead of merely putting your new AI code online in a GitHub or other similar repository, you wrap it in the SingularityNET Agent API, and put it online in a Docker container on a server connected to the Internet? This could be a server you rent oneself, or it could be a server provided by SingularityNET via a hosting partner. Then, once this wrapping is done, others around the world can use your code for a variety of purposes — and they can pay you whatever fee your Agent can negotiate with them.
For other AI researchers who want to download your code and explore it and modify it, the open source code can still be available just as now. But for commercial users who want code to integrate into their applications, it is worth something to be able to simply access your code via a standard API rather than needing to download and fiddle with it. Furthermore, if they are using your code this way, they can expect that you will be maintaining the code and fixing any bugs that arise (and indeed you have an incentive to, since others are using your code and paying you for it).
And then there’s the synergy bonus. The difficulty of working with most existing open source code, also translates into a paucity of work on making various bits of code work together. Once the world’s open source AI tools are wrapped in interfaces making them easy to connect to each other, a lot more experimentation regarding emergent, multi-component intelligence is going to occur. For instance, an intern in our Hanson Robotics / OpenCog lab in Hong Kong recently did some research connecting a neural net algorithm for language processing (Adagram) with some OpenCog scripts for learning syntax from large unlabeled corpora. This was done in the interest of research, as part of a project of ours aimed at making OpenCog learn the grammar of a new language by recognizing patterns in a bunch of examples. As it was, this cross-connection was somewhat tricky, due to the different ways the two pieces of software want to work. If both Adagram and the OpenCog syntax learning script had been wrapped up in SingularityNET nodes, the integration would have been far simpler. Wrapping open source AI code in SingularityNET Agents not only makes it easier to use and monetize, it makes it easier to combine toward the goals of making more powerful commercial software, and making more and more general-scope artificial intelligence.
Just as Uber was the first to majorly tap the hidden-in-plain-sight resource of peoples’ passenger seats, and AirBnB was the first to majorly tap the hidden-in-plain-sight resource of peoples’ empty apartments and bedrooms, so SingularityNET is the first to majorly tap the hidden-in-plain-sight resource of all the open source AI code sitting online in repositories and barely being used.
The challenge is to make the Agent framework easy enough to integrate into, that even AI developers who are “on the fence” about bothering to provide their code to the world for general practical use, decide to take the leap and do it. The SingularityNET software development team — led by Cassio Pennachin, who has been working with me on AI code for 19 years now — is working toward this challenge with energy and inspiration.
As time goes on, a culture will develop around making one’s code available in this way, so that it comes to feel as natural and inevitable putting one’s research code in a repository online for other researchers to play with.
