Benchmarking Dota 2

Benchmarking a game usually means finding the “Benchmark” button somewhere in the options menu and pressing it. Dota 2 does not come with this feature, however. So in order to find a reproducible testing sequence the user has to do that manually.

As of May 2nd 2018, I’ve rewritten parts of the guide to reflect on a new set of methods I’m using right now. Highlights include much more relevant and conscious data and less reliance on the buggy timedemo command.

The Demo Sequence

The perfect benchmark sequence should ideally represent a workload similar to what a player encounters in-game. The problem is, this approach does not necessarily lead to meaningful numbers. While the majority of the time, players might be running around alone, slaying creeps and heroes alike, the worst-case scenario in a game of Dota 2 usually is a large-scale teamfight. This fight pushes the CPU and GPU to the maximum: Massive amounts of hero models, hats, particles and spells flying around and dropping the framerate.

Pewpew

A sufficient demo file proved to be match 3787212636, the first game of the BO3 Semi Finals of the GESC:Jakarta Minor between Infamous and VGJ: Thunder. The teamfight at 13:20 is massive: Nyx Assassin impales, Jakiro icepaths as well as everyone’s favourite Uberlord’s Firestorms, it’s basically particle- and effect galore. There are heavier workloads available but this is a very intense scene. The camera in this script is locked to the Templar Assassin, which also happens to nicely force a bit of water-effects into the benchmark.

The Technical Details

I’ve reworked most of my testing methodology but decided to steer away from the fully automated testing and use a somewhat semi-automated benchmark script with manual input to kick off MSI Afterburner logging.

What we still need:

  • A benchmark.cfg which specifies the match id and a bit of console output to be run

First, download the replay ingame (it’s on the free 2018 GESC: Jakarta ticket) or via Mega and put the 3787212636.dem file into the replays folder in:

Steam\steamapps\common\Dota 2 Beta\game\dota\replays

benchmark.cfg

Navigate to the cfg folder in

Steam\steamapps\common\Dota 2 Beta\game\dota\cfg

and create a benchmark.cfg with the content in this pastebin, or simply download it here.

The .cfg file is mostly inspired (and in parts copied!) by aveyo’s excellent scripts and is navigating the client to the correct time in the replay file and provides a countdown when to hit the begin- and end-benchmark buttons of MSI Afterburner.

benchmark.bat

Create a benchmark.bat file (or download it here) wherever it’s most convenient to you. This script will simply execute Dota several times, tell it to load the benchmark.cfg and simply repeat after the testing was done.

The benchmark.bat has the following content:

@echo onset dota_path=C:\Games\Steam\steamapps\common\dota 2 beta
set renderers=dx11 dx9ex nod3d9ex vulkan gl
set res_x=1920
set res_y=1080
for %%r in (%renderers%) do (
start /wait /d "%dota_path%\game\bin\win64" dota2.exe "-%%r" +con_enable 1 -noborder -windowed -x 0 -y 0 -width %res_x% -height %res_y% +exec_async benchmark +demo_quitafterplayback 1
)

And the settings explanation:

  • dota_path: Points to your Dota 2 install location. You have to change this.

This script launches Dota, executes the benchmark.cfg and shuts Dota down again. The it starts the next test until all renderers are tested.

Please note, due to this being a .bat script, this won’t work on Linux, but a simple .sh script will provide similar functionality.

Collecting frametimes

While the old timedemo command that I used in the older version of this guide provides average fps as well as a variance metric, I now switched to manual benchmarking with MSI Afterburner due to the better data collection. Especially 1% lows are an incredibly useful metric and I personally prefer a smooth game over simply high average fps.

What we’ll need:

  • MSI Afterburner and Riva Tuner Statistics Server: Link

Install it and then do the following in the settings dialogue:

  • In “Monitoring”, check Framerate AVG and 1% and 0.1% lows
MSI Afterburner settings

Verify that RTSS works by simply launching the game. You should have a bunch of information in the top left corner.

Running the benchmark

  1. Run the dota_benchguide.bat file
Ingame output while running the script

The Results

After every finished run, MSI Afterburner dumps the new data in the specified .txt file. Disregard the console output, it’s mostly used as control values (and simply isn’t as thorough as MSI Afterburner’s numbers).

Typical output

The data in the file is in the same order as you specified in your .bat file. So if you went with DX11, DX9, DX9ex, Vulkan, OpenGL order, then that’s the same order that you’ll see the data for.

What I’d suggest, however, is to run the benchmark a few times at least. Especially 0.1% lows are incredibly variable and when I did my RAM benchmarking, I used the average of 5 runs each to calculate the final numbers.

That’s mostly it. You can tinker with the .bat file and the .cfg files a lot to adjust them to your needs. You could try different in-game settings and see how they affect your fps for example.

Lastly, if you’re a member of the tech-press and use these instructions for your content, feel free to reach out if you want to chat about Dota 2 and it’s quirks and/or use this guide to your hearts content if you give credit.

JJ, out.

layerth

Noticeably Invisible.

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store