The current core of Raidbots is Simbot. I’m certainly biased, but I think it’s the easiest and fastest way to use SimulationCraft for most common tasks that players have (generate stat weights and compare gear/relics/talents).
This post aims to provide some clarity and information on what’s happening behind the scenes, why there’s a queue, and what you can expect from the site moving forward. I’ll just cover the high level process, later on I’ll probably write some more technical breakdowns for those that are interested.
The TL;DR
- Scalable — feel free to use the site, it can handle the load.
- Stable — the site will generally only go down when I screw it up, not due to traffic or other shenanigans (and, yes, I know I’m inviting Murphy with these statements).
- Sustainable — Patreon and ads are providing enough funding to run the site at a high level of performance. If income slows down or drops, I can scale back the power to keep the site affordable.
The Basics
Simbot is essentially a website that generates input for Simc, puts that work into a queue, runs SimC on powerful servers, and generates a nice report on the site.
As you enter your character, add gear, tweak item levels, and choose simulation options, the website is just generating the SimC input that you can build by hand. You can even click on “Show SimC Input” below the submit button and see it change in real time. Most of the work I’ve done is to make that process easier — the site talks to Blizzard’s APIs for your character, queries my own simple item database, and generally tries to verify valid input.
When you hit submit, your input is put into a queue. You’ll immediately see where you are in line. If you’re not a patron, your job goes to the back of the line. If you are a patron, you’ll start in front of all the public users but behind other patrons whose sims haven’t been run yet. The length of the line is pretty variable — time of day (Wednesday nights are the busiest), number of worker machines, etc. While there’s no set-in-stone guarantee of wait time, it almost never goes above 5 minutes and is usually closer to 2–3 minutes. You might get lucky and hit a quiet moment where your job starts immediately.
Raidbots uses a dynamic number of workers based on the amount of incoming simulation requests. This is how I can ensure that wait time rarely gets out of control. If work starts building up, Warchief (my behind-the-scenes orchestration server) can spin up more workers. If requests are slowing down, it can spin down workers.
Once your simulation is at the front of the line and a worker machine is ready to run a new sim, you move from the waiting screen to the processing screen. The Progress Donut and SimC output is pretty much the same stuff you can see when running SimC on your own machine. I’ll probably dive deeper into SimC mechanics in the future, but the basic idea for most simulations is that SimC is simulating your character against a virtual boss thousands and thousands of times. Then, based on the kind of simulation you set up, it copies your base character, tweaks your gear, talents, or stats, and then runs again.
Once all the intensive simulating is done, SimC outputs some files, the worker machine moves those into a public location, and the website can show you the report.
The Simbot report page is just a simple reimagining of the incredibly detailed SimC results page (which is also available). These are public and shareable, just send the URL to someone and they can see your report.
Some Details
- Just about everything is fault tolerant. If a worker machine crashes or dies in the middle of your simulation, your job will go back into the queue at the very front of the line and will be restarted on a new worker. Web servers are the same way (though web servers are easier since they are pretty much stateless and only handle very short requests)
- All the servers are in Google Compute Engine (basically Google’s version of AWS). Servers are spread in various US locations. Most of the infrastructure automatically spins up more machines when there is more traffic. I’m also using some of their high availability services like Datastore and Storage to make everything as reliable as possible.
- Given that SimC is incredibly hungry for CPU, I use the fastest machines I can afford. At this time, I’m using 16-CPU machines for workers. These machines are roughly 2–4x as fast as decently specced gaming PCs so once your sim is running you’ll get fast results.
- I try as hard as possible to squash bugs, sim failures, and operational issues quickly before working on new features.
Wrapping Up
I’ll be diving into more technical details at some point in the future but wanted to provide the high level overview to give folks some reference points on how it’s all working.
For the most part, the tagline of the site sums it up — “Run SimulationCraft in the Cloud”. My main job is to get out of the way and let you get answers from SimC.
If you find the site useful, consider becoming a Patron — you can skip to the front of the line and not even worry about the queue time. At the time that I’m writing this, the public queue is generally 2+ minutes while Patrons only wait ~10 seconds for an open slot. You’ll also be ensuring that I can keep running the site at a high performance level!
