Smart Sim is a new option that will produce more precise results, can provide some dramatic speed ups for Top Gear sims, and requires less option fiddling!
Top Gear sims should generally run much more quickly. Other sims may be slower but will be running at a higher precision.
Smart Sim is now live and the default setting on Raidbots, both for the website and the Discord bot. There’s essentially no reason to choose anything else.
Smart Sim + Top Gear
I’ve been a huge fan of the approach that AutoSimc uses to run massive sims and I’ve blatantly stolen that idea and implemented it on Raidbots along with a little bit of added sugar for non-Top Gear sims.
For massive sims the basic idea is to run a series of sims — initially you run very fast but low precision sims, then you choose the winners from that round, increase the precision, and run another round. Eventually you can determine the winner(s) at a high precision without wasting valuable CPU time on bad combinations.
Low precision sims can be run stupidly fast. Instead of requiring 10,000 or more iterations, often only 100–200 iterations are needed to be able to quickly filter out low performing gear sets. The better candidates then get run at a higher precision using more iterations to figure out the next set of winners, and so on.
In my testing so far with Legion itemization, the first round can often eliminate 95% or more of the item sets that have no chance of claiming the highest DPS in a Top Gear sim.
As of now, Raidbots is using 3 stages at 1%, 0.2%, and 0.05% error margins. This is the same default behavior of AutoSimc which seems like it provides good results in culling low performers pretty quickly. I’ll be analyzing results from real world sims and may tweak these values in the future.
Large Top Gear sims (more than 2 million iterations) require Smart Sim to ensure the sim can be finished within a reasonable amount of time.
A Note About Gem/Enchant Sims
One thing to note — sims where you choose multiple gems or enchants to test may take significantly longer to run than ones with just gear/talents.
Gems and enchants tend to provide a much smaller change in your overall DPS which means that more combinations will be at higher precision. Depending on how many sockets you have or how many gems/enchants you choose, Top Gear can take 10x longer (or more) to finish.
In general, I recommend running gem/enchant optimization sims separately from gear/talent runs — you’ll get the results you’re looking for more quickly.
Smart Sim + Everything Else
For non-massive sims (Stat Weights, Gear Compare, basically everything else aside from Top Gear and some large Advanced sims), Smart Sim helps ensure that you’re getting precise results. Instead of using a static iteration amount, it will run using 0.05% margin of error.
This means that if you sim multiple classes and you like to have the same precision on all of your sims, you won’t have to manually fiddle with the Iterations option and/or “waste” iterations in Top Gear by using a higher value than you need (Top Gear and Advanced treat all gear sets as 5k iterations for determining iteration limits).
0.05% target error may be significantly more precise than what the defaults previously provided and so smaller sims may end up taking more time than before. I’ll be listening to feedback very carefully to see if there are strong feelings about speed vs precision from you all.
A Few Technical Notes
I’ll write up a more comprehensive blog post in the future, but the short story is that I’ve been working on significant improvements to the Raidbots backend over the several months to get to this point.
The key approach has been to effectively build my own version of map/reduce which I lovingly call Flightmaster. Flightmaster service instances sit between the web servers submitting simulations and the worker machines — it can handle multistage simulations needed for the AutoSimc approach as well as break down large sims into smaller pieces, run those pieces on the workers, and then reassemble the results into a single file.
The whole project has been super interesting and is making the site a bit more reliable. Parts of Flightmaster have been running on the site for a few months now to make sure I get all the nasty distributed problems worked out before tackling the true goal: larger sims!
Update: Read more about Flightmaster
