Jun 7, 2020
Is that dragon soul worth it? Assessing the significance of neutral objectives in League of Legends via Gradient Boosting Classifier
Playing around neutral monster objectives is integral to League of Legends (League for short) game strategies. Given how complex the game is and the constantly ever-evolving rules, it’s no wonder that players debate over what the most optimal strategies are. Players constantly argue about questions such as:
How important are…
- the first two dragons compared to the rift heralds?
- the third and fourth dragons compared to Baron Nashor (“baron”)?
- the baron compared to the Elder dragon?
To learn about how the current matches are linked with objectives, I used a machine learning classifier, Gradient Boosting Classifier, from the sklearn library. I hope that these findings can help someone else make the most optimal decisions about neutral objectives! Source code on my GitHub.
VERY Brief Overview of Game Knowledge
League is a 5v5 multiplayer online battle arena game (MOBA) where the blue and red teams aim to destroy each others’ nexus. Every game takes place on the same map, Summoner’s Rift (Fig 1) with the blue team on the bottom left side and the red team on the upper right side, and the same monsters. Before the nexus is able to be destroyed, however, there are at least 4 defensive turret structures and 1 inhibitor structure that must be destroyed first. Each nexus spawns waves of minions that will attack enemy minions and structures, and it’s up to the players to manage these minion waves to capture structures. These minions are relatively weak and each team looks to capture neutral objectives to swing the game in their favor.
What are neutral objectives?
Neutral objectives are strong monsters that spawn in alcoves that both teams can access via openings facing the river. These monsters spawn at specific timepoints of the game as well as periodically spawn once an objective is killed. Killing neutral objectives grants unique buffs (in addition to the usual gold and experience) for the team who deals the final blow. The granted advantages may swing the favor of a game or secure a stronger team’s advantages into a victory.
There are 4 major types of objectives spawning at 2 locations on the map.
Baron Nashor pit:
- Before the 20 minute mark when Baron first spawns, the rift herald is available for capture in the baron pit. The rift herald damages and helps in destroying turrets, which also gives gold resource rewards.
- After the 20 minute mark, Baron Nashor spawns and will buff the killing team players and minions temporarily
Dragon pit:
- Elemental dragons grant different advantages depending on the type of dragon: infernal — offensive damage, mountain — defensive stats and shield, cloud-cooldown reduction, ocean — health and mana regeneration. Elemental dragons repeatedly spawn until one team collects 4 dragons total, which will grant that team dragon soul, a much stronger version of the individual dragon buff.
- After dragon soul is taken by a team, elder dragons will spawn instead. Elders are intended to help teams close out games with serious combat buffs. It buffs the teams’ players with extra damage and will automatically execute enemy players that drop below 20% health.
Neutral objectives pose in-game dilemmas on the fly
Notice how the two objective locations are geographically split on the game map, with rift herald and baron spawning on the top side and dragons spawning on the bottom side of the map. The split locations often force teams to prioritize between the two locations; the enemy team is usually able to “trade” by taking neutral or structural objectives on the opposite side of the map. The objective dilemma becomes more significant in professional e-sports, where skilled players are able to utilize small advantages to swing a game in their favor or to snowball leads to victories.
Predicting League of Legends Game Outcomes
Predicting game outcome based on post-game or real-time stats usually involves as many significantly relevant features as possible. These numbers include: player kills, structural objectives captured, gold, experience, items, and neutral objectives. There have been other project looking at the predictive power of very specific features, such as the champions selected by the players, and individual professional e-sports players. I have not found any projects focusing on the importance of objectives nor one that is up-to-date with the latest objective changes for the current season.
This project aims to provide insight on the relative importance of neutral objectives throughout the course of the game, helping players make optimal decisions regarding objectives.
Project Overview
There are two parts to this project:
1. Train a binary classifier model for predicting the winner of a game at different timepoints of a match
2. Assess the relative importance of neutral and structural objectives to predicting game outcomes
— The number of objectives captured up to different timepoints in a game will be used to predict the winner. This will examine the importance of different objective types in predicting the game winner.
— feature importance weights will be extracted from the trained model.
Predictive Models
Two different models were trained to predict the winner of each game, red side or blue side. The models both use only features related to neutral objectives for training and prediction. In contrast, predictive LOL models usually would try to include all relevant statistics such as team gold and kills. However, the focus of this project is to evaluate only specifically the predictive power of one type of resource: neutral objectives.
Model 1:
—Using post-game features: Model 1 used features related to neutral objectives for each team at the end of the game, e.g. number of barons taken by the blue team.
Model 2:
— Analyzed by time Intervals throughout the game: I was interested in the predictive power of neutral objectives at different points of the game, since the value of a dragon varies at different times in the game. At each 5 minute interval during a match, the winner is predicted using neutral objective numbers at that timepoint.
— Streamlined features: the data originally contained two sets of features, one for each team(e.g. number of barons captured by the red team, with the blue team being a separate feature). The two sets of features were combined and expressed as a difference between the two teams, e.g. the difference in number of barons captured between the two teams. This massively streamlines the number of features and increases interpretability with minimal data loss.
Data Extraction from Riot API
The analyzed match data was extracted via the [Riot Games API](https://developer.riotgames.com/apis) and contains over 20,000 NA high elo solo/duo ranked matches from masters, Grandmasters, and Challenger tier players in the North American server on patch 10.11. Riot Watcher API wrapper streamlined the API data extraction process.
Riot API only allows for extraction of match IDs through user accounts, not through elo rankings and times. A list of high elo matches were obtained by extracting summoner names of high elo ranks and compiling a list of their most recent matches. The riotwatcher wrapper API was used to facilitate extracting data.
The match details are obtained from 2 API endpoints:
- match.by_id — gives summary stats of both teams at the end of the game
- match.timeline_by_match — gives detailed information of all players on each team for every minute of the game
Evaluation Metrics
Predicting wins in sports lends itself to gambling applications. Unfortunately, the current model has limited practical applications since it only work mid-game or post-game, while e-sports betting usually happens pre-game. Optimizing for gambling would focus on precision, since maximizing the chance that a winning prediction is correct is critical for minimizing the financial risks of betting on a single game.
Accuracy and F1 score were used as evaluation metrics because they are good general indicators of model performance. Accuracy is the most obvious go-to metric as it represents the proportion of predictions that are correct. Moreover, the classification distribution is balanced and binary, which means the accuracy will most likely not be misleading. F1 score was also used as an indicator to measure and balance precision and recall and help confirm the validity of the accuracy score and robustness of the model.
Recall refers to the number of blue-win games that were correctly classified as such, while precision refers to the proportion of correct predictions (TP) of all blue-win predictions (TP+FP). M. Recall is less important since bets are generally based on individual games. F1 score is also used as a metric to balance precision and recall.
Data Preprocessing
The initial extracted data was stored in dictionaries with the match ID’s as keys. The resulting dictionary file is very large since each timeline provides comprehensive information about the gamestate for each minute interval. This includes player locations, experience points, items, etc.
For the purposes of this project, only details concerning objectives were extracted from the dictionaries. For each match, a Pandas dataframe was used to organize the neutral objectives data, which was then used to generate corresponding features such as the difference in number of dragons taken between the teams. Other than the neutral objectives data, match winner, game duration, first turret, and first inhibitor were also extracted.
After extraction from dictionaries, the dataset is quite easy to work with since the features are either integers (e.g. difference in dragons between the teams) or binary (e.g. team that won). Binary features, with either “blue” or “red” variables, were encoded to 1 and -1 integers before training.
Imputing with means
For this dataset, the NaN values aren’t true missing values; they’re placeholders for 0. The game data only records when a team captures an objective, otherwise the absence of a value means that a team has not captured that objective. However, imputing the mean seems most prudent in order to distinguish between 0 values resulting from the sum of non-zero values and 0 values where both teams have not obtained the objective.
Initial look at match data
Removing “remakes”
The outcome for every match is **always** either red or blue with no missing values. However, the outcome may not always be meaningful — if there are players who never load into the game, their teams are able to surrender within the first few minutes. These games are considered “remakes”, and their outcomes are meaningless even though winners are still assigned. Remakes are available from 3:00 to 4:00 and takes a maximum of 90 seconds, meaning that 5:30 would be the longest time possible for a remake. Therefore the remakes are dropped from the dataset by removing any games under 5:30 mins length (330s).
Quick Look at Game Lengths
The mean match length is 24 minutes with a stardard deviation of 7 minutes. Over 2/3 of the match are between 16:49 and 31:19 minutes in length, and 95% of match are between 9:33 and 38:34. The histogram of match length distribution shows that there is an outlier group of matches at 3–4 minutes. This group represents the remake games explained above.
How common are objective events?
The graph below paints a picture of how prevalent each type of objective is. Inhibitors are the most common event since it is a requirement to winning (assuming the enemy team does not surrender).
The prevalence corresponds to how early the objective can be obtained. Dragons and rift heralds are the most common neutral objectives taken, spawning at 5:00 and 8:00, respectively. Baron is fairly common considering it spawns at 20:00 but is taken in half of the games — a testament to how desirable it is.
In contrast, the late-game objectives, dragon soul and Elder dragon, are rarities. The earliest possible times to obtain these objectives are 20:00 and 26:00, respectively, though realistically these objective events happen later. Each type of soul is obtained in ~5% of games, and since soul events are mutually exclusive, together they are present in ~20% of games. Dragon soul requires at least 4 dragons to be killed, whereas Elder dragon requires one team to obtain dragon soul, so logically Elder dragon is the rarest event captured, present in only 3% of games.
The objective event includes non-neutral objectives, and so any turrets, inhibitors, dragons, etc will count towards this score. No objectives are taken within the first 5 minutes, which is to be expected based on game design. The first neutral objective spawns at 5:00, and turrets might be damaged but are designed to be practically impossible to destroy during the first 5 minutes. By 20:00, however, the vast majority (>90%) of games will have had at least one objective event.
Win Distribution
The outcome data feature is a binary feature that currently holds ‘red’ and ‘blue’ string values corresponding to the teams. Of the 20,154 games in this project’s dataset, red won 51.2% and blue won 48.8% of the games. In comparison, the Blue side generally has a record of being the higher winrate at 53% due to slight asymmetry in the game. The outcome dataset can be considered balanced for the purposes of machine learning models.
Initial correlation heatmaps
Very interesting how high the correlations are for dragon souls with the winner. Infernal soul actually has a correlation coefficient of 1 at 15 minutes. The souls’ association with the winning team decreases past 30 minutes, but is still fairly strongly associated with winning with coefficients between 0.44 and 0.61.
Model Selection
Initial model selection was done by comparing K-Fold Cross-Validation results which identifies Gradient Boosting Classifier as the best performing model.
Model Tuning
RandomSearchCV was subsequently used to tune the model for slight improvements. The tuning was repeated until it resulted in a better model, since the tuning usually resulted in an overfitted model that is less accurate — a common pitfall of ensemble learners.
Parameter distributions used and optimised parameters:
- max_features: range(1, 9) 4
- min_samples_leaf: range(1, 9) 6
- criterion: [“gini”, “friedman_mse”] friedman_mse
- n_estimators:[10, 20, 60, 100] 60
- learning_rate: [0.05, 0.01, 0.1, 0.2, 1] 0.2
- max_depth: [None, 1, 3, 5, 10] 5
As to be expected, prediction accuracy is little better than random during early game when very few objectives are available to be taken, and thus less data available to base predictions upon. As teams capture more objectives throughout the game, the accuracy increases to a maximum of 70.5% at 25 mins and declines thereafter. The decline is also expected since the advantages granted by neutral objectives slowly evens out: the disadvantaged team can use the time in a stalled game to catch up in resources (gold), while gold differences matters less and less as players max out their resource slots.
Extracting Feature Importance
The top 5 most post-game predictive features are:
1. team that takes first inhibitor
- Which team kills the first inhibitor seems to be the best predictor of who will win the game. This makes sense since killing an inhibitor is the most direct path towards killing the enemy nexus. The first inhibitor kill accounts for 40% of the weights for predicting the game winner.
2. elemental dragons difference
- The difference in elemental drakes taken by a team is the next best predictor. Without doing further analysis it’s hard to say whether this difference is statistically significant. We can speculate that if one team is successful at capturing dragon objectives, they likely have a strong control over the game carrying over to wins.
3. game duration — Indicative of the ability for the more advantaged team to close out a game
4. baron kill difference
5. team that takes first baron / team that takes first tower
Different types of objectives seem to be the most valuable (associated with winning) at different points in the game:
- 10 mins: elemental dragon differences, in order of Ocean > Infernal > Mountain > Cloud.
- 15–20 mins: rift herald difference — if the rift herald is captured early enough, it will spawn a second time as a stronger buff. The rift herald will despawn at 20 minutes if not killed, so the window of most significance during 15 and 20 minutes reflects this nicely. The 20 minute association is stronger than the 15 minute one in accordance with the rift herald’s strength.
- 25–35 mins: Baron nashor difference — at 35 mins, soul dragons and elder dragons become more significant, but their effects are still overshadowed by literally every other objective.
Elemental Dragons
It seems that elemental dragon advantages — those found in early game — are much more significantly associated with wins than souls. Certainly part of the reason for the association may be the translation of early game advantages, obtained from outside of neutral objectives (e.g. lane dominance, successful skirmishes), into capturing neutral objectives. There is still the logical possibility of a causal relationship.
The order of significance of the elemental dragons early game could be related to the degree of advantage the dragon bestows on a team, though it’s just as possible that the order reflects prioritization of the dragon types. For example, the ocean dragons are regarded as beneficial for early laning phases, so advantaged team may expend more effort securing the dragon than for a cloud dragon. Considering the priority that teams place on dragons right now as a stepping stone to obtaining dragon soul, the weakness of the individual dragons, and the fact that the soul drake element is not reveal until after the 2nd dragon is killed, it seems likely that the weight differences here stem from the degree of advantages conferred by different types of dragons.
Individiual elemental dragons might be weak (and they are far weaker than in previous seasons), but they are far more significantly related to wins than the objectively stronger elder dragons and dragon souls. The small advantages conferred by individual drakes in the early game might help teams dominate over the enemy team and snowball further advantages. The relatively small importances that souls have towards wins compared to the initial direct correlation might be that by the time these late-game objectives are available, wins are already decided by early game snowballing.
To better establish a causal relationship, Assessing how teams prioritize between the different objectives would be useful to establish a causal relationship.
Future Directions
I had a lot of fun getting this far and there are so many ways I want to expand this project:
- add other relevant match variables: team gold, kills, wards, champion picks, lane positions, etc.
- try fitting a XGBoost Classifier model, a variation of the Gradient Boosting Classifier used for the project
- expand dataset — due to time practicality limitations, only 20,000 matches were extracted from over 60,000 match IDs obtained. Also interested in expanding to other servers— e.g. EU, China, Korean — and to lower elos; masters+ is a very small proportion of the playerbase and may not reflect what the predictors are in lower elo (e.g. Bronze, Silver, Gold)
- evaluate competitive play, which plays out differently than solo queue, and where objective decisions matter more than anywhere else
