Using AI with the Game Tags Classifier

Authors: Louis Keryhuel & Khanh Nguyen

Ready to revolutionize gaming? Discover how Homa’s Game Tags Classifier, with 92% accuracy in predicting game genres, is changing the game market.

Introduction

In this post, we’ll discuss one of our latest projects, the Game Tags Classifier. We were able to create this ambitious proof of concept in just one month, from brainstorming to delivery. If you’re interested in learning how AI can help you predict game genres automatically to monitor the market more efficiently, keep reading!

Part 1: The Context and the Achievement

Every week, thousands of new game apps are submitted to the App Store and PlayStore. As a game publisher and studio, Homa must understand basic app metadata, such as the genre, mechanics, camera angles, and in-game objects. While third-party services can provide this information, they often need more flexibility. For instance, they may not allow us to define our genre categories or tag smaller apps from lesser-known developers. We must keep an eye on our competitors’ prototypes and follow any shift in market trends. We built our metadata prediction tool, the Game Tags Classifier, to solve this problem.

The Game Tags Classifier (GTC) ‘s goal is simple: it allows Homa to identify the genre of hypercasual games available in app stores based on just two inputs: the title and the app’s description.

Despite the success of the project, it took work to build. We faced challenges in choosing the correct methodology for classifying games, selecting the most suitable technology, and determining the source of truth for our input data.

Part 2: The Technology

There are two main considerations when choosing which technology to use for the project.

Type of Data Input for Prediction

There are two main considerations when choosing which technology to use for the project.

Type of Data Input for Prediction

There are two types of possible inputs when looking at the public data available on the app store pages. First, using only text-based information such as the title and description falls into natural language processing. On the other hand, using the screenshot images available would rely heavily on computer vision algorithms to decode the game’s images.

While images can provide much information about a game’s genre, we’ve decided to explore training an AI model using text-based information.

This is because text inputs are much easier to work with compared to images. However, following our literature research, we’ve understood that using images to predict a game’s genre only brings a marginal increase in prediction performance compared to using only text input.

Jiang, Y., & Zheng, L. (2020). Deep learning for video game genre classification. arXiv. https://doi.org/10.48550/arXiv.2011.12143

Prediction Algorithm

For our genre prediction algorithm, we decided to use OpenAI’s GPT3.

At its core, it is an algorithm allowing us to predict a sentence’s following words based on its beginning: Google search’s auto-complete is an excellent illustration. For example, if a user types in “Homa is a mobile game,” the top search suggestion will add the word “company.”
We chose this algorithm due to the following advantages:

🤖 It is a state-of-the-art algorithm that is very performant for many standard language tasks, such as translation, question-answering, or predictive text. This is because GPT3 is built using transformers, neural network architectures that understand natural languages remarkably well.

🤖 Yet, it is flexible enough to be modified based on our own labeled data (games whose genres were manually tagged by our internal team). Afterward, when we provide the AI model with the title and description of a given game, the model can “autocomplete” the game’s genre.

🤖 Lastly, it has a very convenient API for training and prediction, which saves us a lot of time and effort to deploy the model compared to building our own models in-house (using deep learning libraries like TensorFlow or PyTorch).

# Train model
response = openai.FineTune.create(training_file=file_id, model="ada")

# Predict genre of new games using trained model
response = openai.Completion.create(
    model=self.openai_fine_tuned_model_id,
    prompt=input_data,
    max_tokens=1,
    temperature=0,
    logprobs=5,
)

Part 3: The Reliability & the Outcome

Our auto-tagging technology can predict game genres with 92% accuracy.
We managed to achieve this high-accuracy level by following three main steps:

👉 STEP 1: We trained our model with over 20,000 human-tagged apps to define the game genres we wanted to predict. At Homa, we were lucky to already have a large 3-year dataset of manually tagged apps thanks to the past tedious work from our internal market intelligence team. Leveraging this asset helped us tremendously in training our model. Furthermore, using OpenAI to prepare our training dataset only cost us about $5; we were not going to find a better deal to test our prototype.

👉 STEP 2: This trained model was then used for predicting hundreds of untagged games belonging to several genres of the hyper-casual industry, such as ‘Arcade Idle,’ ‘Platformer,’ ‘Defense,’ or ‘Runner,’ for instance. Tagging an app was also very affordable — less than $0.1 per 1,000 tagged apps.

👉 STEP 3: We conducted a ‘human tagging’ assessment to validate the game genre predicted by the Game Tags Classifier. Comparing both sources helped us gain confidence in the output generated by GTC.

Even if some genres were predicted with a higher accuracy level than others, this proof of concept exceeded our expectations.

GTC’s impact on Homa is still felt nine months later as it has fully automated the tedious task of tagging games, allowing our in-house Market Intelligence tools to scale our efforts to identify market trends while keeping a close eye on the game genres our competitors are working on.

Summary

The Technological Choice Was a Primary Factor for Succeeding in This Project

When selecting the technology that would fuel this project, we had a very pragmatic approach, not focusing on overly complex models or trying to achieve 100% prediction accuracy; we’ve decided to focus more on ease of use: using text input instead of images and using GPT3’s API to train and deploy our AI models.

High Impact, Fast Execution

Like everything in Homa, impact mixed with the speed of execution is a crucial value of ours as we were able to build an MVP in under a month, go through small iterations, and build the data pipelines necessary to serve our model’s predictions to both internal dashboards and our Homa Lab data platform.

The Power of Unexplored Data

Working on this project, we realized Homa has accumulated a lot of data just waiting to be analyzed since its inception.

For example, with the painstaking efforts of our Market Intelligence team to manually tag more than 20,000 games over the first three years of the company, we were able to train our models to build this auto-tagging AI technology.

To give some examples of extensive internal datasets with little use today related to marketability & user behavior:

💡 The thousands of games that we tested along the way

💡 The nearly 100,000 creatives lovingly crafted by our creative teams

💡 The more than 1 billion events we receive every single day from our analytics data pipeline

In short, as we’re trying to fully exploit these datasets and more, we’re convinced we can bring many more insights and impact, keeping our competitive edge over the rest of the market.

Conclusion

Game Tags Classifier (GTC) is a powerful proof of concept developed by Homa that uses AI technology, specifically OpenAI’s GPT3, to predict game genres based on text-based inputs such as the title and description of a game app. With an accuracy rate of 92%, GTC has proven to be reliable in identifying game genres in the hyper-casual industry.

Through meticulous training with a large dataset of manually tagged apps and validation through human tagging assessment, Homa has successfully built a genre prediction model that can efficiently monitor the market and keep up with shifts in market trends. This innovative application of AI has allowed Homa to gain a competitive edge in the game publishing and studio industry.

If you’re interested in leveraging the power of AI for your tech job and want to be part of cutting-edge projects like the Game Tags Classifier, we invite you to apply on our website. Join our team at Homa and be at the forefront of technology-driven innovation in the gaming industry.

Visit our website to apply and learn more about our exciting opportunities.

Originally published at https://engineering.homagames.com/using-ai-with-the-game-tags-classifier on 18th of April, 2023.