GPU Benchmarks, CPU Bottlenecks, and The Callisto Protocol Update
The Callisto Protocol, one of the games I use for GPU benchmarking, was just updated to v3.0.1 including “General performance optimizations across all platforms”. This game is very CPU-limited, so I used the opportunity to look at the problem more closely and check if this update fixes it. TLDR: some improvement in performance, but no progress in the CPU bottleneck.
There is minimal change in Native rendering scores, from 72fps to 73fps; but a good bump of +6.55% from 72fps to 77fps with FSR 2 Performance. The new results, both Native and FSR2/P, are identical to what I had in the previous version with GPU overclocking. I also tried the new version with overclocking and the results are now identical to stock: apparently, the game had some optimization in shaders so now it’s never GPU-limited in my system. The brick wall of exactly 77fps in the game engine is still there (the precise limit depends on CPU and RAM speed). To better investigate that, I captured sensor data for the benchmarks:
With or without FSR 2 upscaling, we can notice the problem easily:
- GPU Load (top curve) averages 86%. GPU Power is 310W, well below the 7900 XTX’s stock TDP of 355W. Same numbers for both tests.
- CPU Load (bottom curve) averages 16%. CPU Power is 103W, CPU Max Threads Load 60. Same numbers again for both tests.
These benchmarks execute on an overclocked 5900X, 32GB DRAM 3800MT. I’m not missing CPU power, the game just can’t use it well. Considering idle GPU resources, I could get +14% FPS without the CPU bottleneck.
Finally, the frame-time graph in the middle shows massive jitter around the moving average. At the 10s mark the average is 20ms but individual frames jump from 13ms to 25ms, a range of 2X. Frame time variance summarizes this problem well: for Native rendering, only 78% of frames have <2ms of difference from neighboring frames, 2% are >8ms.
Update: The severe variance in frame-time is much less noticed in normal gameplay, so this is at least in part a problem specific to the built-in benchmark. The performance bottleneck however is the same.
Blaming Ray Tracing?
Perhaps the problem is with Callisto’s advanced RT? Many gamers claim that RT kills framerates and kills puppies, so I had to check.
Without any RT (but all other settings to the max), Native performance shoots up to 121fps and GPU Load now averages 98%. CPU Load is identical at 17%, but the engine pushes a lot more frames with the same CPU work. The frame-time graph is improved from RT, but it’s still poor.
Without RT but with FSR 2 Performance, the CPU bottleneck is still alive: the wall only moved up. I now get 146fps, a very small +21% for rendering at 1:4 scale. We also see GPU Load falling again, now to an average of 96%. CPU Load is only up slightly to 20%, and the frame-time graph is again improved but still not great. At Callisto’s best, Raster + FSR2/P, 85% of frames are <2ms from neighboring frames and 1% are >8ms.
Callisto’s problems are not caused by RT, it’s just a very CPU-limited engine. Any heavy rendering option maximizes the effects of poor scaling. Frame pacing is a worse problem than average performance.
How it’s done: Dying Light 2
The Callisto Protocol features groundbreaking ray-traced graphics, it’s the most visually impressive game I played recently — and that includes Portal with RTX, whose next-gen path tracing puts even an RTX 4090 below 60fps in 1440p without upscaling, but looks like a toy in terms of realism.
I have another game that’s very close in RT goodies: Dying Light 2.
These very different graphs show how rendering is supposed to work:
- GPU Load averages 99% in both tests. It’s actually 100%, with some dips in scene transitions. GPU Power is now 345W.
- Frame-time curves are perfect, zero jitter around the moving averages — exception again for the scene transitions.
- CPU Load is 17%, Max Thread Load 38, Power 89W for Native (52fps); or 32%, Max Thread Load 57, Power 108W for FSR2/P (109fps).
With no bottleneck on the CPU side and plenty CPU power idle, when we reduce rendering resolution with upscaling, the game engine can push a lot more frames per second; consequently, we have more CPU usage.
Dying Light 2 is a great example of frame pacing. It’s so smooth, I honestly can’t see any difference above 60fps. I normally play it with the GPU undervolted and power-limited to -10% (310W) even though overclocking gives it a good increase in FPS. The game doesn’t have a configurable framerate limit, so those tunings are the best way to keep the GPU cooler and the fans quieter. I get the hotspot limited to 72ºC at max RT, FSR2/P.
In my opinion, every reviewer should adopt this game as their go-to benchmark for advanced upscalers: DL2 supports all of FSR 2, XeSS, and DLSS; it’s a ray-traced heavyweight that benefits from this option; and its rendering performance, scaling, and stability are ideal for analysis across a wide range of hardware, resolutions and other factors.
Conclusions
I don’t want to be too hard on Callisto, still very recent and packing significant rendering innovation even compared to DL2, from ray-traced subsurface scattering to feature-movie human model rendering and animation — it’s the first game that closes the Uncanny Valley for me.
But the game engine unfortunately suffers scaling limitations that leave significant performance on the table, at least for high-end GPUs. With my 7900 XTX, I had to resort to FSR 2 Balanced to keep the game above 60fps in the hardest areas at 3440x1440. All rendering settings maxed out, but that’s one of the games where you want to do that if possible. The new patch also adds a New Game+ option, which is great because I couldn’t get even close to filling out the skill/weapons tree despite grinding combat so I could sell all health injectors, etc. But I think I’ll wait for another performance patch or two before going for a second playthrough.
