NVIDIA Shield: reliable graphics?
How does NVIDIA’s flagship Android product fare when put to the GraphicsFuzz ShaderTest?
See also: posts evaluating:
NVIDIA are graphics veterans, and to many people GPU == NVIDIA. So we might expect NVIDIA’s Android graphics drivers to wipe the floor with the competition.
With this in mind, let’s look at how the NVIDIA Shield TV (with a Tegra X1 GPU) and the NVIDIA Shield tablet (with a Tegra K1 GPU) fare on ShaderTest GLES compared with the other devices we have tested so far:*
This table shows the number of issues in each category for each device. What sticks out is that, whilst NVIDIA suffers fewer crashes than ARM and Qualcomm, many tests time out on the NVIDIA devices, something we did not see with the ARM and Qualcomm drivers. This is surprising. Does this mean that NVIDIA’s graphics performance is slower in general, or simply that the tests are exposing performance bugs? If the NVIDIA devices did not time out, would they still end up failing these tests with an error, crash, or rendering issue? The NVIDIA devices also exhibit crashes, rendering issues, and the highest rate of compile failures seen so far.
In summary, NVIDIA is doing better than Qualcomm, but is certainly not wiping the floor with the competition yet!
Do you have a Shield device, or another Android device? Then please try out the GraphicsFuzz Android Benchmark and share your findings on Twitter or Facebook.
In detail
Our website lets you explore our findings for the Shield and Galaxy devices in full. Let’s look at some of the details here.
Timeouts and crashes
ShaderTest GLES runs with configurable time limits for shader compilation and rendering, each set to 30 seconds for our tests. On the Shield TV, 6 variant shaders hit the timeout during compilation, and 8 hit the timeout during rendering (14 timeouts total).
The Shield tablet results are pretty similar, with 11 timeouts total. Notably, the Shield tablet renders correctly for variant 073 of shader family 006, for which the ARM and Qualcomm S8 devices both crash, and the Shield TV times out.
The primary goal of ShaderTest GLES is to assess reliability, and the tests push drivers into tight spots for which high performance is not necessarily critical. Still, it is notable that the Shield devices are slow on these tests compared with the competition. Also, by virtue of not completing these tests in good time, the devices cannot exploit those particular tests to reveal potential rendering issues. Fast shader compilation time is also important to avoid the problem of shader stuttering.
Rendering issues
On the 150 shaders selected from the 3000 that make up ShaderTest GLES, we see three rendering issues affecting the Shield TV, compared with one affecting ARM and six affecting Qualcomm. Two of the Shield TV issues also affect the Shield tablet.
Variant 114 of shader family 010 gives a rendering issue on the Shield TV:
This variant renders fine on the Shield Tablet, showing that there are differences in the graphics drivers for these devices. It also renders fine on the ARM S8, but gives a link error on the Qualcomm S8.
Variant 133 of shader family 006 is handled fine by the S8 devices, but causes a rendering issue on both NVIDIA devices:
The story is similar for variant 212 of shader family 001:
Compile failures
Our tests trigger a number of compile failures on the Shield devices, which turn up as both compile errors and link errors in our results tables.
Some of these are cases of the driver complaining about the syntax of valid shaders, and others are fatal internal errors, for example:
(0) : fatal error C9999: unexpected expression in DUI_foreachIdOne compile error is an instance of the NVIDIA driver doing stricter checking of array accesses than the OpenGL EL shading language specification requires, and NVIDIA might have good reason to favor stricter-than-necessary checking.
Mission impossible?
None of the devices we have looked at so far have been able to successfully process variant 199 from shader family 010. But fear not, we shall see devices that can handle this variant!
Clean sweeps!
We’ve focused so far on bugs and issues highlighted by ShaderTest GLES. But what about successes?
Well, for shader family 003 we see that both NVIDIA Shield devices succeed in a “clean sweep”, rendering images identical to the reference for all variants, while the ARM and Qualcomm GPUs of the Samsung Galaxy S8s suffer from compile failures and crashes.
On the other hand, ARM gets a “clean sweep” for shader family 009. In contrast, NVIDIA suffers from compile failures, timeouts and crashes, and Qualcomm from compile failures, crashes and a rendering issue.
*
The X1 GPU is also used in the Nintendo Switch, which we unfortunately were not able to run ShaderTest GLES on. Please get in touch with us if you can make this happen!
Next time…
…let’s take a look at ShaderTest GLES results for Google’s flagship Pixel 1 and Pixel 2 phones.
See also
Our original series of Medium stories describing the underlying technology.
