DPC Points vs Rating Systems.

With the recent Supermajor direct invite drama, there’s once again a lot of complaints and criticism of the DPC system. Teams, tournament organizers and fans have attempted to use DPC Rankings as justification for specific invites, or as ammunition to attack other invites. I believe that line of argumentation to be quite illogical.

Whilst there are inherent flaws with the DPC points system (sub use, roster abuse, terrible tournament formats, etc), these are beyond the scope of this article — we’re just focusing on direct invites here, and how a solution to that problem can indirectly solve some related issues.

The purpose of the DPC was never intended to be an accurate and up-to-date representation of the various skill levels of teams around the world — it was primarily created as a transparent and objective way to benchmark and measure tournament placement throughout the Dota 2 season so that the Direct Invites to The International would be based entirely on results. This was because teams/players had no way of knowing if they had ‘done enough’ to warrant an invite to a Valve event.

The key issue is that the DPC only values placement within a tournament when allocating points, ignoring the context of the performance of a team in getting that placement. For example, if there was a 16 team single-elimination minor event where Team A beat Virtus.Pro (2–0), Liquid (2–0) & Secret (2–0), losing to EG 1–2 in the finals (who themselves beat 3 no-name teams each 2–1 to get to the finals), then despite the better underlying performance within the event by Team A, they will simply get 2nd place and 90 points per player. When it comes to DPC points this situation is perfectly reasonable, the point distribution itself can be debated — but the underlying principle that Team A simply gets ~90 points here for a 2nd place finish is fine. The bigger concerns for me are that:

  • Virtus Pro might be the 2nd best team overall but never get a chance to place there given their single upset (i.e. bad tournament format and/or bad initial seeding: more here)
  • If future events based direct invites on the pure results of this tournament, there’s a high chance that the seeds are really messed up, leading to somewhat nonsensical data feeding into itself.

A rating system however doesn’t consider an entire tournament as a single entity but rather looks at individual games. This means that winning a bo7 4–0 is better than winning it 4–3 (which is logical). It means that beating 3 strong teams only to lose to another strong team is better than crushing 6 weak teams. It means that a very very strong team beating a very weak team 2–1 is actually a net gain for the weak team. It’s a system which works by negative feedback to ultimately reduce overall error within a system and it’s designed to be not only self-correcting, but still useful/predictive as it’s converging. It’s immune to bad formats, bad seeds and bad pairings. It’s not perfectly predictive, but it aims to be. Every game of the tournament acts as a single data point.

Three commonly used rating systems (Elo32, Glicko, Glicko2) of the top professional teams as of 18th March 2018. The highlighted teams were the direct invites to the PW/PGL Supermajor.

The DPC system isn’t a rating system at all — it’s a placement-based points-centric ranking system. It has very many parameters defining it. It’s not trying to represent skill at a point in time — it’s trying to represent overall performance over an extended period of time. It lacks the corrective measures that rating systems have (since teams can never lose points they can only be overtaken by other similarly-positioned teams scoring points). Attempting to use DPC points as a replacement for a true rating system is just intentionally feeding very noisy data back into itself, and is totally illogical given that there’s not a shortage of way more accurate rating systems available.

Some people have said that this is overly complicated and unnecessary for Dota 2 since ‘the top of the DPC is mostly the top teams in the world (in their opinion)’. There are two low-hanging counterpoints to that:

  • What if a team starts performing badly over a period of time. They can’t drop points, so must we just accept that they’re going to be high seeds who are currently quite low-skilled? If so, it means accepting that the groups / brackets will be skewed.
  • What if teams swap around so totally untested teams form from their players? What if the newly formed teams are much much worse than the previous iteration of teams were?

One other way of justifying a rating system over DPC points when it comes to direct invites and seeds is looking at their predictive power: the underlying principle of direct invites is to reduce variance and chaos within qualifiers; and the point of having seeding coming into an event is to set the format up in a fair way so that the top teams do not end up eliminating each other early (but still allowing truly skilled yet undervalued teams to triumph). Surely if you’re aiming to do this in the best way possible, then the objective and transparent system that most accurately predicts the winner of large number of matches implicitly does the best job at evaluating / seeding the teams in the first place.

One other perk of an objective and well-formed rating system (alongside the DPC system) is that it provides ratings for all teams, not just the ones that have scored points at a DPC event. This makes invites for events a lot easier and more transparent (especially if there’s tiered invites). It means that the reward for performing well in one set of qualifiers (even if you don’t win them) is that you also have slightly easier runs in upcoming qualifiers.

All in all, the DPC system is obviously not perfect — there are various ways in which we all expect it will be improved in the future; but for Tournament Organizers to insist on needlessly compounding the error in it just for the sake of so-called simplicity is utterly ignorant on their part. Fans should not view DPC rankings as a measure of a teams skill (although often there’s a high correlation), but rather as an aggregate metric of how well the team has placed in the DPC season.

~ Noxville / www.datdota.com/ratings

Like what you read? Give Ben Steenhuisen a round of applause.

From a quick cheer to a standing ovation, clap to show how much you enjoyed this story.