Engineering at Dotabuff
Welcome to our engineering blog! This blog is for the technically-minded players and playfully-minded developers, giving you insight into our engineering team and the process behind the features and additions on Dotabuff. In this post I’ll introduce our team and give a brief overview of how everything works at Dotabuff. We’ll try and keep this post up-to-date with any changes. If this type of post interests you, don’t forget to follow the blog.
Our Team
Our engineering team currently consists of three people. It’s smaller than most people guess, and that definitely has pros and cons. This is the team that works on the day-to-day management, engineering, and planning of Dotabuff, We’re supported by a variety of amazing community staff — they help us with everything from social media to UX testing. Our teams are 100% remote and that brings its own benefits and challenges. More on that in another post.
Our People
Sabina (@lawliepop) is our Product Director. Her role is to investigate and develop concepts and ideas that turn into features. Sabina also leads our research and user experience side of the business and helps us understand what our users want and need.
Jason (@jcoene) is our Engineering Lead. His most important role is being our Data Architect, making the important design decisions on how to collect and persist data from dozens of sources. In tandem with that, Jason also manages our hardware and related operations which allows us to store and serve billions of matches with millisecond response times.
Wayne (@watbe), myself, am a Product Engineer. My main role to help implement and maintain key Dotabuff features. I get a lot of freedom to choose what to work on and think up of new things to work on. My main areas are in esports, replay analysis and live matches. That said, both Jason and I are familiar with most of the technical systems (and that means we do take holidays!)
Our Stack
We own and maintain our own hardware — mainly Jason’s area as a result of physical proximity. Our servers run Linux (Ubuntu) and mostly run containers. We use Docker and Chef to manage deployment. The front-end is served by a monolithic Ruby on Rails app but we’re moving more and more of it into our back-end; Go-based micro-services.
We chose Go for it’s strict typing, good performance and low complexity cost. It’s an amazing language for building reliable, predictable, correct systems, I can’t recommend it enough.
— Jason
Our services handle everything from authentication, match ingest, replay analysis, achievements, statistics, rankings, and more much, all independently of each other. Services communicate through a mix of internal HTTP and Protobuf APIs, coordinated by Consul and HAProxy.
We have different strategies for storing and serving information for different areas of Dotabuff. For user data, payment transactions, and other data where using a relational database makes sense, we use PostgreSQL. For match data, player statistics, and other high-volume data, we use Riak and a healthy helping of CRDT logic. Riak allows us to serve data in milliseconds but also meets our needs in other ways. It’s definitely worthy of its own blog post.
For data where lists and rankings make sense, we use Redis. However, Redis is used as a cache and ephemeral data-store, we can always rebuild data on Redis with our other systems. Examples include rankings (heroes, leaderboards, betting, etc), news-feeds, and HyperLogLog cardinalities.
NSQ is used as the core data pipeline. Almost everything piece of externally-sourced data is fetched, queued, and retrieved with our multitude of services to and from NSQ channels. This is what allows us to create and use a service-based architecture with ease.
Other Tools
We use other services/tools as well, either self-hosted or otherwise. Our continuous integration workflow consists of GitHub and Travis-CI. Trello, Slack, Google Calendar, Buffer, and much more are indispensable tools for our remote-based team. Without these tools, nothing gets done!
There’s a few things I’ve missed here (I could write an entire post about TrackDota), and definitely a lot more we can elaborate on. We plan on writing more on decisions we’ve made and will make in the future. There’s a lot of stuff we think are interesting and we hope you find this interesting as well. Don’t forget to follow this blog!
Hiring Soon!
If this type of stuff interests you, we’ll be hiring soon. Keep an eye on our jobs page to see available openings, and make sure you mention this post!
