Throne is a platform for sports prediction using machine learning. It provides users with live competitions, data, features, backtesting modules, and many more features to facilitate the use of quantitative methods in sports. This post introduces Throne, its motivation, and how you can get started.
Sports + Machine Learning
My first exposure to using statistics and machine learning for sports data was as a postgraduate. I needed an application for using non-linear state space methods, which would normally imply a project involving financial time series, but I felt like doing something different. So I looked instead at using these techniques for signal extraction on sports data — how can we extract the ability of teams from data and its evolution over time?
It quickly became clear to me that this interest was less obscure than I had thought, and that there was a fairly established academic interest and commercial interest in the field. Similarly, there was a well established blogging scene in sports analytics. So there was a lot of existing content to work with. But I noticed a glaring problem.
The barriers to entry for making quantitative sports models are high. In particular:
- Data is expensive, both in direct costs and time costs (extraction and maintenance). Data also comes from disparate sources which means building pipelines and performing tedious but crucial ID matching tasks.
- Domain knowledge is important, which means you must invest time in gaining intuition for what are the right tools for the problem. Those who are in-the-know usually have an incentive to stay quiet to maintain any edge they might have in prediction markets.
- Infrastructure is expensive, for example, building an adequate backtesting module to understand the source of your edge and model failings, monitoring your performance against a public benchmark to understand live performance, and calculating daily features so you can predict the latest events.
But given that there was an established interest in predicting sports using statistical methods, it was natural to ask whether these barriers to entry could be reduced to increase accessibility to the field. My answer was Throne: a platform for sports prediction using statistics and machine learning.
The essence of Throne is that it hosts live prediction competitions for sports, including the NFL, NBA, NHL, Premier League and other popular leagues. Participating is as simple as submitting probabilities to the platform, which will then record your performance relative to other users.
Machine learning competitions are not a new concept — and this is not an innovation in itself. But the focus on sports gives Throne’s competitions a particular flavour which is different to existing prediction platforms:
- You are only evaluated on live performance — we give you historical data and some free features, but you aren’t evaluated on a held-out set of data. You are only evaluated on your prediction of upcoming events. This contrasts with platforms which evaluate based on reusable test set.
- Your score is based on relative log loss (cross entropy difference) — we evaluate you relative to a set of publicly available predictions which means you have a direct adversary to compete and compare models against. This is particularly useful for understanding how your model might be failing.
- Feature engineering is paramount — many competitions give you a set of unknown features and require black box algorithms to be trained on them. Black box algorithms can be effective, but Throne emphasises hypothesis generation, which involves designing features with domain knowledge about sport to create models that can find edges against the public.
The live prediction style of the competitions also creates strong incentives for building particular types of models, in particular:
- The production premium — famously the Netflix Prize solution was not actually implemented because it was too hard to put into production. Because Throne evaluates based on live predictions, you will need to build models that are viable for production environments.
- The interpretability premium — what happens if your model starts performing badly for live predictions? Often we need models to be interpretable so that we can find and correct model failings. Black box solutions may still work, but in a live prediction environment, there is increased emphasis on simpler but interpretable models.
- The time-robustness premium — the nature of sports, including rules, strategies and player types, changes over time. Models must account for the non-stationary directly, or be fitted to robust enough patterns to gain a consistent edge. Users must respect the time dimension of the data.
As a platform, Throne also gives you tools to help construct your models:
- Data : we give you full historical results data for each sport usually encompassing 10+ years of results for each league.
- Features : feel free to make your own features, but we have features to get you started. If you make good predictions we’ll reward you with XP that you can use to get our prebuilt features — represented as collectable cards.
- Backtesting : we give you a free backtester so you can evaluate your model’s performance against the public and get a sense where your model might be succeeding or failing.
All of these features are free for our registered users to access.
Of course, if you don’t want to participate in the competitions, but just want to play with some free sports data, that’s great too! Please knock yourself out!
It is easy to register : just link into the website using your existing GitHub account and you are ready to download data, make models and submit predictions. We have a good range of sports and competitions that you can choose from, and we are going to expand the number of competitions in coming months subject to community interest.
We also have a Slack channel where you can discuss the platform and sports prediction more generally with our community, and get tips from other users on getting started with model building.
On a personal note, this has been a project built entirely in spare time in the past 12 months. I am really excited to see how people use the platform, and am also excited for the possibilities that a thriving community focused on sports prediction and analytics may create.