Samsung S9s head-to-head: ARM vs. Qualcomm

Looking at OpenGL ES driver reliabilty with ShaderTest GLES

A snapshot of our results shows a mixture of regressions, persistent bugs and fixes between GPU drivers for Samsung Galaxy S8 and Galaxy S9 devices. Check out the full results.

This is the first in a series of posts where we’ll look at OpenGL ES driver quality on the new Samsung Galaxy S9 phone series, through the lens of ShaderTest GLES.

Android developers, especially those that work on game engines like Unity and Unreal, will be looking for opportunities to offload work to the GPU for better power efficiency and visual effects. But exploiting GPUs to their full potential is impossible if their drivers are unreliable. And so far, ShaderTest GLES exposes bugs in all devices we have tested.

Like its S8 and S7 predecessors, the Samsung Galaxy S9 ships with ARM GPUs in some territories and Qualcomm GPUs in others. Here’s a top-level summary of what our testing efforts show about Galaxy S9 GPU driver quality:

  • Bugs remain: ShaderTest continues to expose a range of driver bugs for both S9 models, but the rate of bugs is slightly lower for the S9s compared with the S8s — it’s great to see that quality is improving!
  • ARM continues to win over Qualcomm: The bug rate for the S9 with a Qualcomm GPU model remains significantly higher than for the model with an ARM GPU.
  • There are regressions: For both GPU vendors we see shaders fail to render on the S9 despite having rendered successfully on the S8. This might suggest that the drivers are changing rather than necessarily improving, or it might suggest that optimisation bugs remain, as these are often difficult to fix and can disappear and reappear depending on the interplay between different optimisations.

As reported by XDA-Developers, we also found an interesting issue where one of our ShaderTest GLES shaders causes the Galaxy S9 model with a Qualcomm GPU to reboot when executed from the Samsung Internet app via WebGL — check out the article here.

Try for yourself

If your device supports WebGL 2 then try out our web app, which will run a selection of tests — let us know if you find any issues! We don’t collect any results, so please do tweet or comment.

Summary of results

This table summarizes the rate of issues found using a subset of 150 tests from ShaderTest GLES’s 3000 tests:

Check out our detailed results for an interactive version of this table.

The devices are ordered by total number of issues found — see our full results for an interactive version of the table. On these tests, significantly more issues are triggered for Galaxy devices with Qualcomm GPUs compared with ARM GPUs, though ShaderTest GLES finds bugs across the board.

The image at the top of the post, repeated below, gives a flavour of the full results:

We highlight, between S8 and S9 drivers, a number of Qualcomm regressions, some bugs that persist in Qualcomm drivers, and an ARM fix.

The ShaderTest GLES tests

The GraphicsFuzz ShaderTest GLES suite offers a rich set of 3000 tests for OpenGL ES fragment shader compilers — one of the most important and intricate components of a graphics driver. Each test checks for similarity of images rendered by two shaders — a reference shader and a variant shader — that are designed to be equivalent by construction. Bugs show up via image mismatches, compile and link failures, crashes, and in the worst case entire system failures.

In upcoming posts we’ll go into a bit more detail on some of the issues that ShaderTest has flagged up.

Check out more in our series of blog posts on ShaderTest GLES