Teach Game-playing Agents to Play Video Games Solely from Audio Cues?

Sound and music are key aspects of video game design, development, and play. They both greatly influence player engagement and investment. Sound provides important gameplay cues. For instance, sound in games can alert a player about an approaching hazard, alert them to collect an item, or even provide clues to help solve certain puzzles.

Learning to Play Video Games from Audio Cues

Game AI Research Group has developed an audio game-playing API within an existing framework for existing agents (and visual-based games). The research is an initial experiment on how game-playing agents can learn to play video games solely from audio cues.

Audio observations in the first 100 ticks of the first level in the game Aliens, played by a random agent. The agent mostly observes feedback on its successful shooting actions (in red), which hit and destroy the protective bases (in orange). One of the avatar’s bullets travels for longer (hence the pause in shooting between ticks 60 and 85) and eventually hits an alien (in blue). One of the aliens also drops a bomb (in green), which doesn’t hit anything. Proximity to the avatar is not taken into account in this example, all sounds are played at full volume.

The researchers expand the Video Game Description Language to allow for audio specification, and the General Video Game AI framework to provide new audio games and an API to teach agents to make use of audio observations. They analyze the games and the audio game design process and include initial results with simple QLearning agents.

Potential Uses and Effects

The suggested framework gives a brief background audio analysis in games. Demonstrated results can be used in combination with other methods to maximize sensor usage for superior game audio performance

This work presents numerous research directions and the researchers have called upon the AI community to do further research in the field.

For instance, the sounds can then be further analyzed to create appropriate responses. they can also be analyzed to help identify what specific sounds might mean. Additionally, the work can be advanced by looking into exactly how certain sounds can be removed and how they can affect an agent’s performance.

Read more: https://arxiv.org/abs/1906.04027

Thanks for reading. Please comment, share and remember to subscribe to our weekly newsletter for the most recent and interesting research papers! You can also follow me on Twitter and LinkedIn. Remember to 👏 if you enjoyed this article. Cheers!