Fireworks’ AI Researchers make history as they combine AI with human cognition

Introduction

Imagine that you have the power to create your own world, in which you will be the main character. What role would you choose if you had this ability? Would you want to go back in time and play like a King of France in the 15th century or a superhero who strives to save humanity, or would you rather play as a hero or evil or something in between from the future who is attempting to conquer space? The possibilities are endless.

In a move that is sure to shake up the gaming industry, Firework Games AI labs have announced the completion of its first milestone, on its aggressive expansion of the Metaverse. By using artificial intelligence to generate unique NPCs (non-player characters), the company is attempting to build a virtual world where you may interact based on your prior activity. The researchers utilized embodiment theory to build a new breed of AI NPCs, and through accelerated deep reinforcement learning AI modules, what might have taken years was accomplished in just a few months.

Users’ Metaverse experiences will be more personalized and immersive as a result of this innovation. The company’s goal is to create simulations on-demand based on users’ previous behavior, to pre-emptively anticipate and tailor simulations to their specific desire. Ranging from NPCs’ behavior, color, and style of environment theme are just a few of the factors.

The company is combining the use of Accelerated Deep Reinforcement Learning Artificial Intelligent Module and Embodiment hypothesis on game development. This will allow AIs to construct more realistic environments and human-like characters. This combination enables Firework Games to create more life-like, and immersive experiences on both games and social applications. The accelerated deep reinforcement learning artificial intelligence module will be used to train AI using synthetic data. Synthetic data is data that is generated by algorithms rather than being collected from real-world sources. This will allow AIs to learn faster and be more accurate in their representation of the world.

The researchers opted for the westward expansion and settlement of the American West in the late 1800s as a setting since it resonates with HBO’s famous series “Westworld.” An illustration of how Human and Artificial Intelligence play out.

Deep Dive into the unknown

What is the Embodiment hypothesis?

The Embodiment hypothesis claims that the mind is not only embodied but also extended in the world. The hypothesis has its roots in artificial intelligence and cognitive science, and it has been used to explain a wide range of phenomena, from simple motor tasks to social cognition. Our AI labs’ have been researching how artificial intelligence can be used to create more realistic environments and human-like characters.

Approach for implementation

High-level overview:

1. Allow a neural network to be inserted into the virtual world (instance) and have human senses added to the AI, and the AI will then engage with a variety of items and settings in the environment.

○ Output: The AI’s sensory input allows us to teach it to interact with a broad range of items and environments, label, and even produce synthetic data while improving the AI’s capacity to generate comparable settings. The AI searches and labels objects in the virtual world automatically.

2. The AI will be trained in a variety of locations and games simultaneously to learn intelligent behavior through accelerated deep reinforcement learning. The AI’s neural network may comprehend human behavior and react realistically to humans in real-time as a result of exposure to human behavior. The more it is exposed to humans, the closer it gets to looking and acting like one.

○ Expected results: realistic human-level NPCs will be generated, as well as behavior. The AI’s capacity to produce comparable yet distinct settings is enhanced by the addition of different environments. At this stage, we anticipate that AI-controlled NPC actions will be indistinguishable from regular people.

3. Though the AI’s objective will be defined by the director, they have a well-defined degree of freedom to be creative. However, in the end, the narrative will adhere to the director’s script. The game’s story can be changed at any time by the game director, and different endings may result based on the AI’s goals. As more user data is taken into account, the AI’s scope of creativity will grow and the story depicted by the AI will be more sophisticated.

○ Expected results: the director can modify the storyline and have different ending scenarios for each NPC. The NPCs will automatically take reasonable action to fulfill their mission given their personality, their interaction with humans, and other NPCs. This will enable a whole new approach to creating games. The game will have a high value of re-playable as every round of the game will expose the player to a new story path.

The ongoing experiment

The ongoing experiment comprises two main parts:

The first element is the assertion that the mind is embodied, implying not only that it isn’t a brain in a vat, but also that it is intimately connected to the body. The second component claims that the mind is extended, which means it makes use of environmental clues to guide its choices.

One of the key features is that it explains how artificial intelligence can be used to create more realistic environments and human-like characters. By using environmental cues, AIs can create NPCs that are more believable and life-like.

1. Environment: The westward expansion and occupation of the American West in the late 1800s. We chose this timeline as it resonates with the famous TV show “Westworld”. Another advantage is that because this world provides a never-ending variety of scenarios for players, we can train the AIs so they don’t overfit to one scenario, ensuring that players have an infinite number of alternate environments during this era.
2. Characters’ sensorimotor: Initially the characters have their own personalities based on the story script. The AI is equipped with the vision and the sonic is to interact with the original characters and the environment: the characters observe the environment and listen to the surrounding environment. The AI will then change their view based on the action of other characters or humans and what they listen to, changing their perception of the people they interact with.

Figure 1. The focus view of AI

Figure 2. The depth image above from AI gives the sense of three-dimensional space.

Figure 3. Normal mapping, which gives the texture, through texture, we create shaders.

Figure 4. Sound map visualization is implemented in the environment. When the AI detects the sound, it will hear it. It will gradually categorize with various sounds and respond to what the reward is based on what it hears.

The team believes that their work will have a significant impact on how artificial intelligence is used in games and other simulations. The use of AI in games has been increasing steadily over the past few years. However, most AI-powered games still rely on pre-scripted events and simple rules to create their gameplay. This research represents a significant step forward in the use of AI to create more realistic and believable virtual worlds.

This new approach is revolutionary since it establishes a common thread that would tie together otherwise unrelated pieces of content, and construct an entire game using only this technique, providing a finished product that reflects one’s personal style and theme. Although there may be more than one way to do so, depending on your skill level as well as your choice of tools, we can generate as many simulations as needed. Allowing players to engage in the simulation of their desire.

Firework Games’ core is its innovative team, with experts from various backgrounds (former Tencent and NetEase game designers and artists, artificial intelligence, virtual reality, and major financial institution like executive director of investment bank and Ex trader of top hedge fund) aiming to write a new chapter in not only gaming but also the metaverse. Unlike other startups, Firework Games has a research division at its inception stage; the company aims to step foot among the giants and establish a distinct niche for itself.

About Us

Firework Games is a decentralized self-development and release platform for blockchain games. The infrastructure is designed to facilitate access to blockchain-based games and build a bridge for traditional games to evolve into blockchain games. Firework Games envisions lowering the barriers for developers to build games by providing technical support, platform support, funding, and community governance

Official Links: Twitter | Telegram | Discord | Website | Metaverse