Google and Qualcomm: Pixel-Perfect?
Looking at Android graphics driver reliability for Google’s Pixel phones
See also: posts evaluating:
The Google Pixel 2 and Samsung Galaxy S8 are widely considered to be two of the best Android handsets currently on the market. The Pixel 2 and S8 US model both have Qualcomm Adreno 540 GPUs, so we might expect their graphics to be equally reliable.
Actually, not quite: querying for GPU driver details shows that the Pixel 2 ships with a fairly recent Qualcomm driver (October 2017), while the S8 ships with an older driver (March 2017).
We saw in a previous post that the Qualcomm S8 driver is unreliable compared with the driver for its ARM S8 counterpart. Does the more recent driver in the Pixel 2 fix these issues?
It seems not: if anything, the update makes matters worse. Here is a summary of the issues found by the subset of ShaderTest GLES tests that we have been using to evaluate devices:
The Pixel 2, and its Pixel 1 predecessor (with a Qualcomm Adreno 530 GPU) rank bottom if we sort by the total number of issues flagged by the tests. Our interactive table lets you sort by other criteria.
As we discuss in “the details” below, the driver update introduces some new compile and link errors, trades some rendering issues for crashes and vice-versa, and introduces more new failures than it fixes.
Of course, our tests are just one lens through which to view reliability. It is likely that the newer Pixel 2 driver also fixes some important issues that our tests do not flag up. But our results underline the fact that GPU driver regressions are common! Our hope is that ShaderTest can help by quickly flagging regressions before a driver update ships.
Pixel 1 vs. Pixel 2
The drivers for the two Pixel phones behave very similarly on our tests. There are a few differences, though:
- Two crashes on the Pixel 2 that render fine on the Pixel 1 — variant 073 of shader family 006 and variant 122 of shader family 009
- A Pixel 1 crash that instead gives a compilation error on the Pixel 2 — variant 065 of shader family 008. The compiler error string is empty, however.
- A compilation timeout on the Pixel 1 for variant 046 of shader family 005; this renders fine on the Pixel 2.
- A bad image on the Pixel 1 that shows up as a crash on the Pixel 2 — variant 051 of shader family 008; however, we have observed this variant to fluctuate nondeterministically between crashing and misrendering on Qualcomm drivers:
- A fix: a shader that gives bad images on both the Pixel 1 and the Galaxy S8, but gives the right image on the Pixel 2; variant 050 of shader family 009:
Pixel 2 vs. S8
Comparing the Pixel 2 and S8 results in more detail, there are a number of apparent fixes in the more recent Pixel 2 Qualcomm driver, including variant 050 of shader family 009 (see above), and variant 228 of shader family 007, which gives a link error on the S8 but renders fine on the Pixel 2.
However, the newer Qualcomm driver exhibits several additional compilation and linking failures on the Pixel 2 compared with the older S8 driver. In some cases, the relevant variant shaders render just fine with the S8 driver —variant 065 of shader family 008, once again, and also variant 127 of shader family 007.
An instance of one bug type being traded for another is flagged up by variant 092 of shader family 009: the variant crashes with the S8 driver, and renders without crashing on the Pixel 2 driver. However, the resulting image is unexpected:
What is less bad, an explicit crash, or a silent miscompilation? The former is more directly disruptive, but the latter could pose a greater debugging challenge for developers, who may believe their code is wrong when really an issue is due to a miscompliation.
Check out variant 199 of shader family 010: in our posts so far we have yet to see a device that successfully renders the expected image. But, once again, fear not: we shall in due course!
…we’ll look at a Nexus device with a PowerVR GPU from Imagination Technologies.