Recently I had a great conversation with Josh Menke. Josh has worked for many years in the gaming industry at RIOT games, Activision, 343, Blizzard etc. Here are some interesting lessons I learned that could be applicable to any industry/organization trying to advance AI.
What are you optimizing?
It is critical to define the right objective function for your AI system. Josh has worked at ‘Match making’ systems for many years. In multi-player games, teams (of multiple players), play against each other. In order for a player to be part of a team, the AI algorithm needs to find a match. If matches are fair, then players will stick around.
Inputs can be: How good is the player. How good other players are. And how similar other teams will be, etc.
But the key question is what are you optimizing? If one were to build a system that accurately predicts which team will win, and use that to assign players; then the game may not be ‘fun’ for the players. They may not feel that the game is fair. And ultimately will not engage/come back.
The most natural thing to optimize is player retention. But that is hard to use as an objective function for the core AI algorithm. One very simple architectural pattern used to solve this issue is to build two layers.
- AI Engine
- Business Rules Engine
The AI engine can predict the outcome of each match. And a rule/decision engine can optimize combining these ‘matches’ for player retention. One such very simple depiction is shown here:
In such a system AI system can further predict ‘contributing variables’ such as Win/Loss rate, Coins earned, ‘Kill Rate’ etc. And these could be further used in optimizing Matches for 7-day retention of 14-day retention of the players.
Cold Start
In every AI system, a very standard problem is ‘cold start’. How to predict any of these outcomes for a ‘new’ player. Someone who does not have historical data to learn from. One interesting idea that Josh shared is most gaming companies have new players play against a bot. The Bot improves its own game in response to the player. And in that process captures data points for the player that can be used for the actual match.
Preparing for Humans Gaming the systems!
There has been a lot of discussion on how AI would take over humanity. I do not know how close AI is to that stage. But we still have many AI systems that are ‘gamed’ by the humans using them. One example from the gaming industry is smart players figuring out how the system does match-making. And gaming it to get ‘easy’ matches for themselves.
Players routinely ‘test’ the match-making system: ‘what if I fake being bad for 10 games, then will the system match me against bad players so I can slaughter players for the next 20 games?’ A typical approach used to solve this is ‘Anlomoly Detection’. Very commonly used in identity systems to find Account takeover.
There are many other fascinating things I learned from my call with Josh! That included advances in deep learning ( Autoencoders with LSTM), advances in Unsupervised Learning, and in general machine learning for optimal match-making. More on that in a future article!
