Benchmarking Dota 2

Benchmarking a game usually means finding the “Benchmark” button somewhere in the options menu and pressing it. Dota 2 does not come with this feature, however. So in order to find a reproducible testing sequence the user has to do that manually.

As of May 2nd 2018, I’ve rewritten parts of the guide to reflect on a new set of methods I’m using right now. Highlights include much more relevant and conscious data and less reliance on the buggy timedemo command.


The Demo Sequence

The perfect benchmark sequence should ideally represent a workload similar to what a player encounters in-game. The problem is, this approach does not necessarily lead to meaningful numbers. While the majority of the time, players might be running around alone, slaying creeps and heroes alike, the worst-case scenario in a game of Dota 2 usually is a large-scale teamfight. This fight pushes the CPU and GPU to the maximum: Massive amounts of hero models, hats, particles and spells flying around and dropping the framerate.

Pewpew

A sufficient demo file proved to be match 3787212636, the first game of the BO3 Semi Finals of the GESC:Jakarta Minor between Infamous and VGJ: Thunder. The teamfight at 13:20 is massive: Nyx Assassin impales, Jakiro icepaths as well as everyone’s favourite Uberlord’s Firestorms, it’s basically particle- and effect galore. There are heavier workloads available but this is a very intense scene. The camera in this script is locked to the Templar Assassin, which also happens to nicely force a bit of water-effects into the benchmark.


The Technical Details

I’ve reworked most of my testing methodology but decided to steer away from the fully automated testing and use a somewhat semi-automated benchmark script with manual input to kick off MSI Afterburner logging.

What we still need:

First, download the replay ingame (it’s on the free 2018 GESC: Jakarta ticket) or via Mega and put the 3787212636.dem file into the replays folder in:

Steam\steamapps\common\Dota 2 Beta\game\dota\replays

benchmark.cfg

Navigate to the cfg folder in

Steam\steamapps\common\Dota 2 Beta\game\dota\cfg

and create a benchmark.cfg with the content in this pastebin, or simply download it here.

The .cfg file is mostly inspired (and in parts copied!) by aveyo’s excellent scripts and is navigating the client to the correct time in the replay file and provides a countdown when to hit the begin- and end-benchmark buttons of MSI Afterburner.

benchmark.bat

Create a benchmark.bat file (or download it here) wherever it’s most convenient to you. This script will simply execute Dota several times, tell it to load the benchmark.cfg and simply repeat after the testing was done.

The benchmark.bat has the following content:

@echo onset dota_path=C:\Games\Steam\steamapps\common\dota 2 beta
set renderers=dx11 dx9ex nod3d9ex vulkan gl
set res_x=1920
set res_y=1080
for %%r in (%renderers%) do (
start /wait /d "%dota_path%\game\bin\win64" dota2.exe "-%%r" +con_enable 1 -noborder -windowed -x 0 -y 0 -width %res_x% -height %res_y% +exec_async benchmark +demo_quitafterplayback 1
)

And the settings explanation:

This script launches Dota, executes the benchmark.cfg and shuts Dota down again. The it starts the next test until all renderers are tested.

Please note, due to this being a .bat script, this won’t work on Linux, but a simple .sh script will provide similar functionality.


Collecting frametimes

While the old timedemo command that I used in the older version of this guide provides average fps as well as a variance metric, I now switched to manual benchmarking with MSI Afterburner due to the better data collection. Especially 1% lows are an incredibly useful metric and I personally prefer a smooth game over simply high average fps.

What we’ll need:

Install it and then do the following in the settings dialogue:

MSI Afterburner settings

Verify that RTSS works by simply launching the game. You should have a bunch of information in the top left corner.

Running the benchmark

  1. Run the dota_benchguide.bat file
  2. Dota will now open
  3. When the console output counts you down, start the benchmark by pressing the hotkey you set in MSI Afterburner
  4. When the console outputs counts you down again, stop the benchmark by pressing the other hotkey you set in MSI Afterburner
  5. Dota will now close
  6. If you specified multiple render options in your .bat file, it will loop back to step 2 now until all settings are done (possibly 5 times if you benchmark all 5 different render paths)
Ingame output while running the script

The Results

After every finished run, MSI Afterburner dumps the new data in the specified .txt file. Disregard the console output, it’s mostly used as control values (and simply isn’t as thorough as MSI Afterburner’s numbers).

Typical output

The data in the file is in the same order as you specified in your .bat file. So if you went with DX11, DX9, DX9ex, Vulkan, OpenGL order, then that’s the same order that you’ll see the data for.

What I’d suggest, however, is to run the benchmark a few times at least. Especially 0.1% lows are incredibly variable and when I did my RAM benchmarking, I used the average of 5 runs each to calculate the final numbers.


That’s mostly it. You can tinker with the .bat file and the .cfg files a lot to adjust them to your needs. You could try different in-game settings and see how they affect your fps for example.

Lastly, if you’re a member of the tech-press and use these instructions for your content, feel free to reach out if you want to chat about Dota 2 and it’s quirks and/or use this guide to your hearts content if you give credit.

JJ, out.

layerth

Noticeably Invisible.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store