Practical Artificial Intelligence applications for game development
Artificial Intelligence (AI) has progressed with supersonic speed in recent years.
Big game companies like EA and Ubisoft are actively integrating these machine-learning and deep-learning techs into their development pipeline. DeepMind and OpenAI have used reinforcement learning to train AI agents play Starcraft and Dota 2, and beat top-notch human players. Nvidia uses deep-learning to achieve real-time ray-tracing, and further improve the performance through DLSS. There is a consensus that AI will revolutionize the game industry.
It may feel that those sexy AI tools are only relevant to the big companies and not very accessible to us common folks. Fortunately, that’s not true.
I have actually come across quite a lot of interesting and exciting AI applications that can facilitate game development, and definitely can be of help for developers from small studios or even indie developers.
I would like to share my two cents and introduce these practical AI applications. By “practical”, I mean these applications are
- readily available as a web app or executable
- free!
Let’s dive in!
Practical AI applications for 2D assets generation
GAN (Generative Adversarial Network) has become the cornerstone for image generation. And in the world of GANs, StyleGAN is no doubt the new king. It has pushed the boundary of the generated image quality, and it is almost impossible to distinguish between a generated face image v.s. a real one. This great website uses StyleGAN model to serve you a person’s portrait image whenever you refresh the page, and you can feel free to use it in your game without violating any person’s portrait rights because, well, that person in the image does not exist.
If reality is not your cup of tea, you may try this. Again this uses the StyleGAN tech. In addition to serving you a generated anime character portrait, it uses a language AI model to generate a backstory of the character.
And it does not stop with human and anime, this website compiles a list of things that “does not exist”. Definitely a good place to get materials and inspirations.
Although these tools generate very good quality images, it may not be easy to find the exact match in your mind as the images are generated randomly.
As human beings, we crave more control, and the good news is that there are conditional GANs, which means the generation is based on some inputs provided by the user.
Here is a tool generating anime character based on user’s inputs including hair color, hair type, eye color, etc.
Or you can try this: provide a photo as input, to get an anime character looks like that person in the photo.
And if you like doodling out your idea, you should definitely try this. It can be a great tool for quick concept art.
Practical AI applications for 2D assets editing
You probably have heard of DLSS (deep learning super sampling), which is a technology introduced by Nvidia. By turning on DLSS in gaming, you can get higher resolution graphics with minimum impact on frame-rate. The core algorithm in this technology is super-resolution using deep-learning. This same algorithm can also help during game development to increase the resolution of a low-res/blurring texture.
But take note that the performance of these learning-based tools is very dependent on the dataset they have trained on. If your image is of anime/cartoon type, this may give good results as it is trained on similar images. While a realistic type of image probably gets a better result using this or this.
Another popular AI application in image editing is “Inpainting”. The vanilla usage of this tech is to restore a damaged image, in other words, remove defects from an image. As it is up for the user to define what is a “defect”, the application actually feels more like a smart photoshop where you can interactively remove any unwanted stuff from an image. A good place to try this is here.
Another great application is neural style transfer, which empowers you to apply a fanciful style of another image to your image. There are many online apps doing this, and by far, I found this to be the best among them.
Moreover, recently there is this DAIN (Depth-Aware Video Frame Interpolation) which can convert any animation/video to a higher frame-rate. Not long after the release of the research paper, someone has built a downloadable version of it (DAIN-APP). Look at how people used it to transform everything into 60fps!
But what if you only have a static image and do not have the budget or time to do any animation at all? Our team has identified this pain point and is developing a tool to facilitate easy animation production. It is not “readily available” yet, but our team is working hard to this out. Keep an eye if you are interested!
Practical AI applications for 3D assets
While GANs have proven to be effective in the 2D world, it is another difficulty level when it comes to 3D. Although there are many active research projects going on for 3D modeling, texturing, rigging, and animation, most of them are still in the research stage, and can be tricky to fit into the actual game development process.
Nonetheless, there are still some practical one out there. One of them is the lip-syncing plugin by Oculus. It can be integrated with both Unity and Unreal, and auto-animate your character’s lip according to the voice input.
Here is another cool application that generates a 3D face model with texture based on a single image.
Practical AI applications for coding
You have probably heard of GPT-3 in the news, and in case you haven’t, this is a new language deep learning model created by OpenAI to produce human-like text. What is special is that it has a gigantic capacity (17 billion parameters in the model) and a gigantic training dataset (entire English Wikipedia is only 0.6% of its training data), so that it can handle almost every language-related task, ranging from generating fake news to writing functional code.
While we are not ready to hand over the entirety of our programming job to an AI model, we can use its help to not only syntactically, but also semantically auto-complete our code. Both TabNine and Kite offers this technology. I’m using TabNine, and the “tab” key soon becomes my favorite key when coding.
Conclusion
There are many more exciting applications I did not list down because they have not reached the product phase, and not considered as “practical” yet.
Just to list down a few potential tools:
- Detect 3D pose and movement from a video— cheap MoCap!
2. Blend different parts of different characters to form a new character.
3. Depth estimation from a single image, which enables occlusion supported AR application.
Given the mushrooming speed of progress in this field, these applications, together with many more, will probably be available just in a few months’ time.
I have faith that AI technology will enable us to offload more and more tedious work to the machine so that we can focus on the creative and interesting stuff. By harnessing the power of AI, a small game development team can overcome resource constraints and limitations, and there will be more indie game developers being able to showcase their brilliant ideas.
