An Apple sandwich
Is graphics more reliable on newer iPhones?
In our posts so far about ShaderTest GLES we have focused exclusively on Android devices. The GraphicsFuzz benchmark also lets you try some of these tests out on your Android device directly in your browser.
Although Apple’s preferred API for graphics on iOS and macOS is Metal, iOS devices also support OpenGL ES. In fact, as we shall see later in the post, it appears that Apple implement OpenGL ES by wrapping Metal.
So let’s now look at results for our tests on a selection of iPhone models, from the iPhone X right back to the iPhone 6.
An ARM-Apple-Qualcomm sandwich
If we rank by total issues found, our results so far place the Huawei Honor 10 phone, with an ARM GPU, top in terms of reliability. At the bottom is the Google Pixel 1 phone, with a Qualcomm GPU. Our interactive table lets you rank the results according to each type of issue.
To put the reliability of these iPhone models in perspective, this page shows results for the iPhone 6, 7, 8 and X models, sandwiched between most and least reliable, i.e. between results for the Honor 10 (ARM) and Pixel 1 (Qualcomm). Here’s a summary of the results for this “sandwich”:
Newer iPhones: lots more rendering issues!
The first thing that sticks out from our results is that we see a lot of rendering issues — cases where a different image from the reference image was rendered — on the iPhone 8 and iPhone X.
In particular, there are a lot more rendering issues on these devices compared with the iPhone 6 and 7 models. For the iPhone 6 and iPhone 7, we see 5 rendering issues total for each device.
In contrast, the iPhone 8 and X models exhibit the highest rate of rendering issues seen yet in the devices we have tested, with 17 of the 150 test shaders that we are reporting on in these posts triggering rendering issues:
iPhone 6–7 and iPhone 8-X — same GPU; same driver?
Another thing that sticks out strongly from our results is that the results for the iPhone 6 and 7 are identical to one another, and the same goes for the iPhone 8 and iPhone X results. The 6 and 7 both ship with an Apple A10 GPU, while the 8 and X have an A11 GPU; our results suggest that those models with the same GPU perhaps share identical drivers.
Complementary rendering issues
If we focus in on rendering issues there is almost no overlap between the 6/7 and 8/X models. Most of the rendering issues overall affect the 8/X models and render fine on the 6/7 models. An exception is variant 212 from shader set 001, which renders a bad image across all the iOS devices we tested.
Common crashes when lowering to Metal
There are quite a few cases where we see a crash on all the iOS devices — for example, variants 021, 056 and 175 of shader family 004.
In these cases, the crash is at compile-time, and it is the MTLCompilerService process that crashes, which is part of the Metal framework at:
/System/Library/Frameworks/Metal.framework/XPCServices/MTLCompilerService.xpc/MTLCompilerServicewith the crash logs showing information related to segmentation faults such as:
Exception Type: EXC_BAD_ACCESS (SIGSEGV)
Exception Subtype: KERN_INVALID_ADDRESS at 0x0000000000000010Our impression is that these crashes all occur in Apple’s GLSL front-end, and are presumably related to lowering GLSL constructs to Metal. We guess that this lowering code doesn’t change much, if at all, between GPU versions, hence why these crashes occur across all devices.
Crashing vs. rendering wrongly
Variant 145 from shader family 010 shows a case where a wrong image is rendered on the older iPhone models, while rendering crashes on the iPhone 8 and X models. What would you prefer: a shader that causes a crash, or a shader that is just rendered plain wrong?
Info from the crash logs associated with render-time crashes again suggest that OpenGL ES is implemented by wrapping metal:
Thread 0 name: Dispatch queue: com.apple.main-thread
Thread 0 Crashed:
0 libsystem_kernel.dylib 0x00000001811802e8 __pthread_kill + 8
1 libsystem_pthread.dylib 0x0000000181299748 pthread_kill$VARIANT$armv81 + 360
2 libsystem_c.dylib 0x00000001810eefbc abort + 140
3 AppleMetalGLRenderer 0x00000001a0d1f0f4 GLDContextRec::flushContextInternal+ 94452 () + 740
4 GLEngine 0x00000001a1b2e01c gliPresentViewES_Exec + 188
5 OpenGLES 0x0000000184531a8c -[EAGLContext presentRenderbuffer:] + 80
6 GLKit 0x000000018e54f988 -[GLKView _display:] + 308
7 GLKit 0x000000018e550b50 -[GLKViewController _updateAndDraw] + 520A little bit of nondeterminism
Variant 122 from shader family 003 shows another interesting discrepancy between the older and newer models: rendering is nondeterministic on the 6/7 models, with slight differences between successive frames:
while rendering crashes on the 8/X models, with the crash logs including:
Exception Type: EXC_CRASH (SIGABRT)
Exception Codes: 0x0000000000000000, 0x0000000000000000
Exception Note: EXC_CORPSE_NOTIFY
Triggered by Thread: 0Summary
Compared with the competition, Apple’s older iPhone models do pretty well on our set of tests. But ShaderTest highlighting a host of rendering issues for the more recent iPhone 8 and iPhone X models.
It would be great to see these issues fixed, and ShaderTest used to ensure that they never come back!
The GraphicsFuzz benchmark doesn’t work on iOS devices, which don’t support WebGL 2. But if you’ve got a WebGL 2-compatible device, such as an Android phone, then try it out!
