Open Virtual Beings for the Open Metaverse

Eibriel
Eibriel
Published in
5 min readJan 30, 2022

We all dream to befriend silicon based characters. But how should those creatures be designed and built? What should be their purpose?

Here I’ll describe a framework to build decentralized, interoperable, and secure open virtual beings.

“Free Guy” a movie showing kind and empathetic virtual beings

Support my work by wishlisting Compicactus on Steam

Here are some basic characteristics for virtual companions:

  • Should run on the client side, without the need of a server
  • All the user information should be encrypted
  • The companion should be able to learn and reason about facts
  • The companion should work on any platform

⚠️ For this article I am assuming that a Virtual Being is a character able to communicate trough natural language, and have permanence of state (memory).

Client side

Many virtual beings require an internet connection, a remote server takes our requests and process them. But that is not actually needed.

Virtual beings can be small programs that easily fit in a mobile phone memory (I’m not including speech to text and text to speech technology, although soon that will be in the reach of any smart phone).

The user could voluntarily send anonymous feedback to help improve the behavior of the virtual being, and receive regular updates.

We can take a look to Kuki, for example, one of the most popular chatbots, has only a size of 50Mb on disk.

💡To achieve this I propose to use a knowledge based system, and clever tricks in the user interface to limit the user input. No deep learning models for now, since they require a lot of parallel processing power, and some times a lot of memory.

Encryption

The information can be stored on IPFS, using Textile and Ceramic, and encrypted using our Web3 identity. There are platforms already using this technology, like the social network Orbis.

Another amazing technology is zero knowledge proof, that will allow the virtual being to make requests in behalf of the user (for example calling a cab) without reveling any personal information (sounds like magic, but is real!)

This also ensures that the virtual being don’t have any hidden agenda, it will always respond to interests and values aligned to the user.

The virtual being will be not only able to keep your secrets, but will be impossible for it to communicate those to others.

💡I propose IPFS, Ceramic and others, to store the state of the virtual being, and to be shared between several clients (desktop, mobile) and virtual worlds (Decentraland, Webaverse).

Learning and reasoning

Many modern virtual beings lack the ability to store facts in memory and reason about them. At most they can remember the name of the user.

But they can’t, for example, know that an user that is over 18 years most probably don’t go to school anymore.

But that is a solved issue since ages ago, with the creation of Interactive Fiction. Those are games, simulations of fairly complex environments, and the interaction between elements.

Some interesting tools use complex logic to describe and infuse life to this small worlds: Inform7 allows the author to describe the simulation in natural language, and Versu can represent complex social rules, and make characters adhere to them.

Promotional material for Versu, “where readers become characters”

One interesting general framework is OpenAgent, described on the book Linguistics for the age of AI (short description). Proposing to use intermediate representations for textual, visual and others input and output.

Those agents, also called LEIAs, should be able to understand natural language (at least in many circumstances), keep track of events, and build plans to achieve goals.

This architecture can also learn from the user, and could even learn by itself.

Here is a demo, by Ivan Leon (password: leia-demo-2019), where a human and an agent work together to build furniture.

💡I propose the use of the OntoAgent cognitive architecture, in fact I’m already doing some experiments on building my own open source version of OntoAgent.

Multiplatform

Should be possible to load any virtual being, using the same code base, on any platform. In the same way a PDF, or a JPG can be loaded on any PC, notebook or mobile phone. They should also able to work in any virtual world, be it Decentraland, Webaverse or VRChat.

There are a few ways of achieving this:

  • Building a standard file specification, with many interpreters that can read it
  • Compiling the virtual being as a piece of software, and create many virtual machines to run it

Creating virtual machines is complex, and are banned from some app stores. In the other hand having a file specification and allowing people to create different interpreters may result in fragmentation, were some virtual beings are only compatible with some interpreters.

💡 This point needs further analysis, but I think the file specification is the way to go. Many interpreters can use the same Node library, for example, to keep the improvements in a single codebase.

An avatar across many virtual worlds, by Jin

Conclusion

In conclusion I propose the following guidelines to create the AI for decentralized, interoperable, and secure virtual beings:

  • Use a knowledge-based approach for AI, with some aid from machine learning and no or minimal aid from deep learning.
  • Use the powerful new tools allowing to store information in a decentralized and private way. The “shared” knowledge will be stored publicly, and the “user generated” knowledge will be stored encrypted.
  • Use specifically the OntoAgent cognitive architecture. Since its designed to have robust natural language understanding capabilities, work on virtual (or real) environments, and take action (complete goals).
  • And finally create a file specification that can be loaded to any interpreter, no mater the hardware, operative system, or software used as a substrate.

Current state

I’m currently developing an open source implementation of OntoAgent as a node library to serve as a foundation for a reference implementation for the Open Beings: https://github.com/Eibriel/CompiAgent

Thank you for reading! 🎉

If you are interested in the topic please DM me on Twitter: https://twitter.com/EibrielBot

Eibriel
eibriel.eth.limo

--

--

Eibriel
Eibriel
Editor for

Interactive media creator, Artificial Intelligence researcher. Slightly byslexic.