The Distributed Atomspace (DAS): A New-age Knowledge Repository
On April 30th, 2024, we hosted a landmark ‘Technical Tuesdays’ session to celebrate the Alpha release of OpenCog Hyperon.
The session introduced several key components of OpenCog Hyperon and announced their readiness for further use, including the Distributed Atomspace (DAS) and the MeTTa (Meta Type Talk) language interpreter.
In the session, our panel of SingularityNET experts, featuring CEO Dr. Ben Goertzel, Chief AGI Officer Dr. Alexey Potapov, and CSO Dr. Matt Ikle, shared more information about the Distributed Atomspace, and how this new-age knowledge repository will empower a new wave of developers to propel Artificial General Intelligence (AGI) research forward.
They highlighted how DAS, an advanced, decentralized knowledge representation and reasoning system, was primed for further exploration and ultimately designed to scale efficiently across distributed computing environments. This capability is fundamental when it comes to managing the vast and intricate datasets required for developing AGI.
DAS’s innovative architecture supports a wide array of cognitive processes, enabling seamless collaboration and data sharing across diverse nodes in a network. This system underpins the neural-symbolic integration and probabilistic reasoning that are foundational to Hyperon’s approach to AGI.
Today, we aim to provide a high-level overview of DAS, as well as share more information about the transformative potential of this groundbreaking technology within the context of OpenCog Hyperon.
Please remember: to learn more about the technicalities behind DAS, please visit the official DAS GitHub repository. To find out about the larger picture of how DAS, MeTTa, and the overall vision of OpenCog Hyperon fit together, visit the OpenCog Hyperon website and the Hyperon GitHub repository.
Introduction to the Distributed Atomspace (DAS)
At its core, Atomspace is a hypergraph used by OpenCog Hyperon to represent and store knowledge. It serves as the primary knowledge source for AI agents, and it encapsulates any computational results achieved during their execution.
The Distributed Atomspace extends this concept into a more autonomous component — it is designed to support multiple simultaneous connections with various AI algorithms, offering a flexible query interface to distributed knowledge bases.
DAS can function as a Python library component or a stand-alone server, capable of storing vast knowledge bases. It enables agents to traverse hypergraph regions and execute global queries involving properties, connectivity, subgraph topology, and more. Whether utilized locally or remotely, DAS provides a consistent API for querying or traversing the Atomspace.
The DAS Architecture
The architecture of DAS can be understood through its main components: the Traverse Engine, Query Engine, Cache, and AtomDB. Each of these components plays an important role in the functionality and efficiency of DAS.
Traverse Engine
The Traverse Engine is responsible for hypergraph traversal. It interacts with the Query Engine and Cache to allow users to navigate the Atomspace hypergraph. Operations such as finding links pointing to or from a specific atom or identifying atoms in the surrounding neighborhood are handled by this engine. It also manages the pre-fetching of surrounding atoms when using a remote DAS, ensuring rapid link following.
Query Engine
The Query Engine processes global queries, including pattern matching. Users can specify whether queries should consider only local atoms or include remote DASs. For remote queries, the engine connects to OpenFaaS servers, combining local and remote information to provide comprehensive results. For example, if different versions of the same atom exist locally and in a remote DAS, the local version is returned.
Cache Layer
DAS incorporates a sophisticated cache layer to accelerate queries involving remote DASs. Unlike traditional caches, the DAS cache not only stores data but also sorts and partitions query results, presenting the most relevant results first. Queries returning more than one atom provide an iterator to the results, fetching additional chunks on demand. This design optimizes relevance and efficiency, both important factors for AI agents performing combinatorial searches.
AtomDB
AtomDB serves as a Data Access Object, abstracting database calls where atoms are stored. This abstraction allows changes or extensions to the actual data storage without impacting query algorithms. AtomDB can be backed by in-RAM data structures or one or more DBMSs, providing flexibility and scalability.
Higher-Level Indexing
DAS leverages DBMS indexing capabilities and custom indexes, notably the Pattern Inverted Index.
This index maps patterns to their occurrences in the knowledge base, akin to document retrieval systems’ inverted indexes.
For instance, a link like Inherits <Concept A> <Concept B> can be indexed to facilitate efficient pattern matching for queries involving subpatterns like Inherits <Concept A> $1, Inherits $1 <Concept B> or Inherits $1 $2.
Pattern Matcher
The Query Engine’s pattern matching capabilities enable DAS to answer complex queries. These queries involve specifying patterns or boolean expressions of subpatterns, which the engine matches against the knowledge base. For example, finding two nodes linked by a similarity relation but not sharing a common ancestor involves pattern matching, providing valuable insights for AI applications.
Mapping Knowledge Bases to Nodes and Links
Ultimately, before loading a knowledge base into DAS, proper mapping to Atomspace nodes and links is essential. DAS does not assume predefined atom types or provide built-in import mechanisms for various formats. Currently, only MeTTa knowledge bases are supported. However, a new component is under development to allow the creation of DAS on top of existing SQL databases without the need to explicitly convert and load data.
DAS Server Deployment and Architecture
DAS server deployment follows a Lambda Architecture using OpenFaaS or AWS Lambda. OpenFaaS is prioritized, with deployment involving Redis and MongoDB or their AWS equivalents. Functions are deployed as Docker containers, managed by a CI/CD pipeline and stored in a private Docker registry. Clients can connect via HTTP, gRPC, or external lambda functions.
Bridging DAS and MeTTa
A significant milestone in DAS development was the integration with MeTTa using Space API in late 2023. This integration bridges DAS’s powerful knowledge representation capabilities with MeTTa’s dynamic reasoning and pattern-matching strengths, enhancing the overall functionality and application range of both systems.
MeTTa, the programming language designed for OpenCog Hyperon, integrates seamlessly with the Distributed Atomspace. MeTTa allows for the creation of introspective and self-modifying programs that can leverage the distributed nature of DAS for enhanced performance and scalability. MeTTa can interact with the Distributed Atomspace to fetch, store, and manipulate atoms. This interaction is optimized for distributed processing, allowing developers to write high-level code without worrying about the underlying distribution mechanics.
The combination of the Distributed Atomspace (DAS) and the MeTTa programming language creates a robust framework for developing Artificial General Intelligence by seamlessly integrating scalable, decentralized knowledge representation with powerful, introspective programming capabilities.
Together, they enable the creation of sophisticated AI systems that can learn, reason, and adapt in real-time, leveraging the distributed computing power to handle complex queries and data-intensive applications.
This synergy between DAS and MeTTa forms a foundational infrastructure that supports the development of AGI, capable of advancing beyond current limitations in both performance and scalability.
One notable application of DAS is in the Bio-Atomspace toolkit, used by Rejuve.Bio for longevity research. This toolkit integrates and analyzes vast amounts of biological data, deriving actionable insights for aging interventions. By leveraging DAS and advanced AI technologies, Rejuve.Bio aims to transform aging from an inevitable decline into a manageable aspect of life, pioneering new strategies for health and longevity. DAS can also be used to power the cognitive functions of robots and virtual agents. For example, a robot using DAS can store and process sensory data, action plans, and learned behaviors across multiple nodes, enhancing its ability to learn and adapt in real-time.
The Distributed Atomspace represents a significant advancement in knowledge representation and storage for AI systems. Its robust architecture provides a flexible, efficient, and scalable solution that is ultimately required to move forward on the journey to developing AGI. With applications spanning from dynamic reasoning to longevity research, DAS is set to be a cornerstone in the development of advanced AI systems. As DAS continues to evolve, so does its seemingly unlimited potential to revolutionize knowledge repositories in the world of AI.
About SingularityNET
SingularityNET was founded by Dr. Ben Goertzel with the mission of creating a decentralized, democratic, inclusive, and beneficial Artificial General Intelligence (AGI). An AGI is not dependent on any central entity, is open to anyone, and is not restricted to the narrow goals of a single corporation or even a single country. The SingularityNET team includes seasoned engineers, scientists, researchers, entrepreneurs, and marketers. Our core platform and AI teams are further complemented by specialized teams devoted to application areas such as finance, robotics, biomedical AI, media, arts, and entertainment.
Decentralized AI Platform | OpenCog Hyperon | Ecosystem | ASI Alliance
Stay Up to Date With the Latest SingularityNET News and Updates:
- Visit our Website
- Subscribe to our Newsletter
- Join our Discord
- Follow us on X
- Follow us on Mindplex Social
- Join our Main Telegram Channel
- Subscribe to our Announcements channel on Telegram
- Join our WhatsApp broadcasting channel
- Follow us on Instagram: SingularityNET and Visions of AGI
- Follow us on Warpcast
- Explore our Jobs Page → We’re hiring.
- Add the SingularityNET Community Events Calendar to stay current on Community, Ambassador, and Deep Funding Events!
