Samsung S9s head-to-head: ARM vs. Qualcomm
Looking at OpenGL ES driver reliabilty with ShaderTest GLES
This is the first in a series of posts where we’ll look at OpenGL ES driver quality on the new Samsung Galaxy S9 phone series, through the lens of ShaderTest GLES.
Android developers, especially those that work on game engines like Unity and Unreal, will be looking for opportunities to offload work to the GPU for better power efficiency and visual effects. But exploiting GPUs to their full potential is impossible if their drivers are unreliable. And so far, ShaderTest GLES exposes bugs in all devices we have tested.
Like its S8 and S7 predecessors, the Samsung Galaxy S9 ships with ARM GPUs in some territories and Qualcomm GPUs in others. Here’s a top-level summary of what our testing efforts show about Galaxy S9 GPU driver quality:
- Bugs remain: ShaderTest continues to expose a range of driver bugs for both S9 models, but the rate of bugs is slightly lower for the S9s compared with the S8s — it’s great to see that quality is improving!
- ARM continues to win over Qualcomm: The bug rate for the S9 with a Qualcomm GPU model remains significantly higher than for the model with an ARM GPU.
- There are regressions: For both GPU vendors we see shaders fail to render on the S9 despite having rendered successfully on the S8. This might suggest that the drivers are changing rather than necessarily improving, or it might suggest that optimisation bugs remain, as these are often difficult to fix and can disappear and reappear depending on the interplay between different optimisations.
As reported by XDA-Developers, we also found an interesting issue where one of our ShaderTest GLES shaders causes the Galaxy S9 model with a Qualcomm GPU to reboot when executed from the Samsung Internet app via WebGL — check out the article here.
Try for yourself
If your device supports WebGL 2 then try out our web app, which will run a selection of tests — let us know if you find any issues! We don’t collect any results, so please do tweet or comment.
Summary of results
This table summarizes the rate of issues found using a subset of 150 tests from ShaderTest GLES’s 3000 tests:
The devices are ordered by total number of issues found — see our full results for an interactive version of the table. On these tests, significantly more issues are triggered for Galaxy devices with Qualcomm GPUs compared with ARM GPUs, though ShaderTest GLES finds bugs across the board.
The image at the top of the post, repeated below, gives a flavour of the full results:
We highlight, between S8 and S9 drivers, a number of Qualcomm regressions, some bugs that persist in Qualcomm drivers, and an ARM fix.
The ShaderTest GLES tests
The GraphicsFuzz ShaderTest GLES suite offers a rich set of 3000 tests for OpenGL ES fragment shader compilers — one of the most important and intricate components of a graphics driver. Each test checks for similarity of images rendered by two shaders — a reference shader and a variant shader — that are designed to be equivalent by construction. Bugs show up via image mismatches, compile and link failures, crashes, and in the worst case entire system failures.
In upcoming posts we’ll go into a bit more detail on some of the issues that ShaderTest has flagged up.
Check out more in our series of blog posts on ShaderTest GLES
- Not all Galaxies are made equal: A summary of Samsung Galaxy S6, S7 and S8.
- An Apple sandwich: Is graphics more reliable on newer iPhones?
- ARM GPUs in Huawei phones: Android OpenGL ES support is getting better!
- Nondeterminism on PowerVR: Could garbage renderings leak information?
- Google and Qualcomm: Pixel-Perfect? When a driver update makes things worse!
- NVIDIA Shield: reliable graphics? NVIDIA Shield TV and Tablet reliability results available!
- Samsung brawl: S8 versus S8: The Samsung Galaxy S8 GPU clash: ARM vs Qualcomm!
